{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 任意选一个你喜欢的整数，这能帮你得到稳定的结果\n",
    "seed = 9999"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 欢迎来到线性回归项目\n",
    "\n",
    "若项目中的题目有困难没完成也没关系，我们鼓励你带着问题提交项目，评审人会给予你诸多帮助。\n",
    "\n",
    "所有选做题都可以不做，不影响项目通过。如果你做了，那么项目评审会帮你批改，也会因为选做部分做错而判定为不通过。\n",
    "\n",
    "其中非代码题可以提交手写后扫描的 pdf 文件，或使用 Latex 在文档中直接回答。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 矩阵运算\n",
    "\n",
    "## 1.1 创建一个 4*4 的单位矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 这个项目设计来帮你熟悉 python list 和线性代数\n",
    "# 你不能调用任何NumPy以及相关的科学计算库来完成作业\n",
    "\n",
    "\n",
    "# 本项目要求矩阵统一使用二维列表表示，如下：\n",
    "A = [[1,2,3], \n",
    "     [2,3,3], \n",
    "     [1,2,5]]\n",
    "\n",
    "B = [[1,2,3,5], \n",
    "     [2,3,3,5], \n",
    "     [1,2,5,1]]\n",
    "\n",
    "# 向量也用二维列表表示\n",
    "C = [[1],\n",
    "     [2],\n",
    "     [3]]\n",
    "\n",
    "#TODO 创建一个 4*4 单位矩阵\n",
    "I = [\n",
    "    [1, 0, 0, 0],\n",
    "    [0, 1, 0, 0],\n",
    "    [0, 0, 1, 0],\n",
    "    [0, 0, 0, 1]]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 返回矩阵的行数和列数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 返回矩阵的行数和列数\n",
    "def shape(M):\n",
    "    if not isinstance(M[0], list):\n",
    "        row = 1\n",
    "        col = 0\n",
    "    else:\n",
    "        row, col = len(M), len(M[0])\n",
    "    return row,col\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.002s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 shape 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 每个元素四舍五入到特定小数数位"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 每个元素四舍五入到特定小数数位\n",
    "# 直接修改参数矩阵，无返回值\n",
    "def matxRound(M, decPts=4):\n",
    "    rows_M, cols_M = shape(M)\n",
    "    for row in range(rows_M):\n",
    "        for col in range(cols_M):\n",
    "            val = M[row][col]\n",
    "            M[row][col] = round(val, decPts)\n",
    "    pass\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.037s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 matxRound 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_matxRound"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 计算矩阵的转置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 计算矩阵的转置\n",
    "def transpose(M):\n",
    "    '''\n",
    "    zip()函数的原理：\n",
    "    zip([iterable, ...])函数接受一系列可迭代的对象作为参数，将对象中对应的元素打包成一个个tuple（元组），\n",
    "    然后返回由这些tuples组成的list(列表)。 利用*号操作符，可以将list unzip(解压)。\n",
    "    \n",
    "    所以，zip(*M)将会把矩阵M的列解压成行，但是每个行都是tuple, \n",
    "    再通过列表推导式的方法对每行从tuple转成list,即可实现矩阵的行、列互换实现转置。\n",
    "    \n",
    "    '''\n",
    "    return [list(col) for col in zip(*M)]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.014s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 transpose 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_transpose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 计算矩阵乘法 AB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 计算矩阵乘法 AB，如果无法相乘则raise ValueError\n",
    "def matxMultiply(A, B):\n",
    "    rows_A, cols_A = shape(A)\n",
    "    rows_B, cols_B = shape(B)\n",
    "\n",
    "    if not cols_A == rows_B:\n",
    "        raise ValueError(\"Matrix A's columns must be equal to B's rows\")\n",
    "\n",
    "    #创建一个0矩阵，列数等于B的列数，行数等于A的行数\n",
    "    result = [[0] * cols_B for i in range(rows_A)]\n",
    "    for i in range(rows_A):\n",
    "        for j in range(cols_B):\n",
    "            for k in range(rows_B):\n",
    "                result[i][j] += A[i][k]*B[k][j]\n",
    "\n",
    "    return result\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.055s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 matxMultiply 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_matxMultiply"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# 2 Gaussign Jordan 消元法\n",
    "\n",
    "## 2.1 构造增广矩阵\n",
    "\n",
    "$ A = \\begin{bmatrix}\n",
    "    a_{11}    & a_{12} & ... & a_{1n}\\\\\n",
    "    a_{21}    & a_{22} & ... & a_{2n}\\\\\n",
    "    a_{31}    & a_{22} & ... & a_{3n}\\\\\n",
    "    ...    & ... & ... & ...\\\\\n",
    "    a_{n1}    & a_{n2} & ... & a_{nn}\\\\\n",
    "\\end{bmatrix} , b = \\begin{bmatrix}\n",
    "    b_{1}  \\\\\n",
    "    b_{2}  \\\\\n",
    "    b_{3}  \\\\\n",
    "    ...    \\\\\n",
    "    b_{n}  \\\\\n",
    "\\end{bmatrix}$\n",
    "\n",
    "返回 $ Ab = \\begin{bmatrix}\n",
    "    a_{11}    & a_{12} & ... & a_{1n} & b_{1}\\\\\n",
    "    a_{21}    & a_{22} & ... & a_{2n} & b_{2}\\\\\n",
    "    a_{31}    & a_{22} & ... & a_{3n} & b_{3}\\\\\n",
    "    ...    & ... & ... & ...& ...\\\\\n",
    "    a_{n1}    & a_{n2} & ... & a_{nn} & b_{n} \\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 构造增广矩阵，假设A，b行数相同\n",
    "def augmentMatrix(A, b):\n",
    "    #列表相加返回的结果实际上是列表拼接\n",
    "    #用列表推导式和zip()可轻松实现增广矩阵的拼接\n",
    "    return [ra + rb for ra,rb in zip(A,b)]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.003s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 augmentMatrix 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_augmentMatrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 初等行变换\n",
    "- 交换两行\n",
    "- 把某行乘以一个非零常数\n",
    "- 把某行加上另一行的若干倍："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO r1 <---> r2\n",
    "# 直接修改参数矩阵，无返回值\n",
    "def swapRows(M, r1, r2):\n",
    "    M[r1], M[r2] = M[r2], M[r1]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.002s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 swapRows 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_swapRows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO r1 <--- r1 * scale\n",
    "# scale为0是非法输入，要求 raise ValueError\n",
    "# 直接修改参数矩阵，无返回值\n",
    "def scaleRow(M, r, scale):\n",
    "    if scale == 0:\n",
    "        raise ValueError(\"Scale parameter must be non-zero value\")\n",
    "    for i,ele in enumerate(M[r]):\n",
    "        M[r][i] = ele * scale\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.004s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 scaleRow 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_scaleRow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO r1 <--- r1 + r2*scale\n",
    "# 直接修改参数矩阵，无返回值\n",
    "def addScaledRow(M, r1, r2, scale):\n",
    "    for i, ele in enumerate(M[r2]):\n",
    "        M[r1][i] = M[r1][i] + ele * scale\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.004s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 addScaledRow 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_addScaledRow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3  Gaussian Jordan 消元法求解 Ax = b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.1 算法\n",
    "\n",
    "步骤1 检查A，b是否行数相同\n",
    "\n",
    "步骤2 构造增广矩阵Ab\n",
    "\n",
    "步骤3 逐列转换Ab为化简行阶梯形矩阵 [中文维基链接](https://zh.wikipedia.org/wiki/%E9%98%B6%E6%A2%AF%E5%BD%A2%E7%9F%A9%E9%98%B5#.E5.8C.96.E7.AE.80.E5.90.8E.E7.9A.84-.7Bzh-hans:.E8.A1.8C.3B_zh-hant:.E5.88.97.3B.7D-.E9.98.B6.E6.A2.AF.E5.BD.A2.E7.9F.A9.E9.98.B5)\n",
    "    \n",
    "    对于Ab的每一列（最后一列除外）\n",
    "        当前列为列c\n",
    "        寻找列c中 对角线以及对角线以下所有元素（行 c~N）的绝对值的最大值\n",
    "        如果绝对值最大值为0\n",
    "            那么A为奇异矩阵，返回None (你可以在选做问题2.4中证明为什么这里A一定是奇异矩阵)\n",
    "        否则\n",
    "            使用第一个行变换，将绝对值最大值所在行交换到对角线元素所在行（行c） \n",
    "            使用第二个行变换，将列c的对角线元素缩放为1\n",
    "            多次使用第三个行变换，将列c的其他元素消为0\n",
    "            \n",
    "步骤4 返回Ab的最后一列\n",
    "\n",
    "**注：** 我们并没有按照常规方法先把矩阵转化为行阶梯形矩阵，再转换为化简行阶梯形矩阵，而是一步到位。如果你熟悉常规方法的话，可以思考一下两者的等价性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.2 算法推演\n",
    "\n",
    "为了充分了解Gaussian Jordan消元法的计算流程，请根据Gaussian Jordan消元法，分别手动推演矩阵A为***可逆矩阵***，矩阵A为***奇异矩阵***两种情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   7,   5,   3,  -5 ||  1  \n",
      "  -4,   6,   2,  -2 ||  1  \n",
      "  -9,   4,  -5,   9 ||  1  \n",
      "  -9, -10,   5,  -4 ||  1  \n"
     ]
    }
   ],
   "source": [
    "# 不要修改这里！\n",
    "from helper import *\n",
    "\n",
    "A = generateMatrix(4,seed,singular=False)\n",
    "b = np.ones(shape=(4,1)) # it doesn't matter\n",
    "Ab = augmentMatrix(A.tolist(),b.tolist()) # please make sure you already correct implement augmentMatrix\n",
    "printInMatrixFormat(Ab,padding=4,truncating=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请按照算法的步骤3，逐步推演***可逆矩阵***的变换。\n",
    "\n",
    "在下面列出每一次循环体执行之后的增广矩阵。\n",
    "\n",
    "要求：\n",
    "1. 做分数运算\n",
    "2. 使用`\\frac{n}{m}`来渲染分数，如下：\n",
    " - $\\frac{n}{m}$\n",
    " - $-\\frac{a}{b}$\n",
    "\n",
    "增广矩阵\n",
    "$ Ab = \\begin{bmatrix}\n",
    "    7 & 5 & 3 & -5 & 1\\\\\n",
    "    -4 & 6 & 2 & -2 & 1\\\\\n",
    "    -9 & 4 & -5 & 9 & 1\\\\\n",
    "    -9 & -10 & 5 & -4 & 1\\end{bmatrix}$\n",
    "\n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & \\frac{-4}{9} & \\frac{5}{9} & -1 & \\frac{-1}{9}\\\\\n",
    "    0 & \\frac{38}{9} & \\frac{38}{9} & -6 & \\frac{5}{9}\\\\\n",
    "    0 & \\frac{73}{9} & \\frac{-8}{9} & 2 & \\frac{16}{9}\\\\\n",
    "    0 & -14 & 10 & -13 & 0\\end{bmatrix}$\n",
    "    \n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & 0 & \\frac{5}{21} & \\frac{-37}{63} & \\frac{-1}{9}\\\\\n",
    "    0 & 1 & \\frac{-5}{7} & \\frac{13}{14} & 0\\\\\n",
    "    0 & 0 & \\frac{103}{21} & \\frac{-69}{126} & \\frac{16}{9}\\\\\n",
    "    0 & 0 & \\frac{152}{21} & \\frac{-625}{63} & \\frac{5}{9}\\end{bmatrix}$\n",
    "    \n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & 0 & 0 & \\frac{-119}{456} & \\frac{-59}{456}\\\\\n",
    "    0 & 1 & 0 & \\frac{-23}{456} & \\frac{25}{456}\\\\\n",
    "    0 & 0 & 1 & \\frac{-625}{456} & \\frac{35}{456}\\\\\n",
    "    0 & 0 & 0 & \\frac{181}{152} & \\frac{213}{152}\\end{bmatrix}$\n",
    "\n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & 0 & 0 & 0 & \\frac{193}{1086}\\\\\n",
    "    0 & 1 & 0 & 0 & \\frac{62}{543}\\\\\n",
    "    0 & 0 & 1 & 0 & \\frac{1835}{1086}\\\\\n",
    "    0 & 0 & 0 & 1 & \\frac{213}{181}\\end{bmatrix}$\n",
    "\n",
    "$...$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  -7,  -3,   1,  -9 ||  1  \n",
      "   0,   0,   0,   0 ||  1  \n",
      "  -2,   7,   7,  -3 ||  1  \n",
      "   8,  -5,  -6,   3 ||  1  \n"
     ]
    }
   ],
   "source": [
    "# 不要修改这里！\n",
    "A = generateMatrix(4,seed,singular=True)\n",
    "b = np.ones(shape=(4,1)) # it doesn't matter\n",
    "Ab = augmentMatrix(A.tolist(),b.tolist()) # please make sure you already correct implement augmentMatrix\n",
    "printInMatrixFormat(Ab,padding=4,truncating=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请按照算法的步骤3，逐步推演***奇异矩阵***的变换。\n",
    "\n",
    "在下面列出每一次循环体执行之后的增广矩阵。\n",
    "\n",
    "要求：\n",
    "1. 做分数运算\n",
    "2. 使用`\\frac{n}{m}`来渲染分数，如下：\n",
    " - $\\frac{n}{m}$\n",
    " - $-\\frac{a}{b}$\n",
    "\n",
    "增广矩阵\n",
    "$ Ab = \\begin{bmatrix}\n",
    "    -7 & -3 & 1 & -9 & 1\\\\\n",
    "    0 & 0 & 0 & 0 & 1\\\\\n",
    "    -2 & 7 & 7 & -3 & 1\\\\\n",
    "    8 & -5 & -6 & 3 & 1\\end{bmatrix}$\n",
    "\n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & \\frac{-5}{8} & \\frac{-3}{4} & \\frac{3}{8} & \\frac{1}{8}\\\\\n",
    "    0 & 0 & 0 & 0 & 1\\\\\n",
    "    0 & \\frac{23}{4} & \\frac{11}{2} & \\frac{-9}{4} & \\frac{5}{4}\\\\\n",
    "    0 & \\frac{-59}{8} & \\frac{-17}{4} & \\frac{-51}{8} & \\frac{15}{8}\\end{bmatrix}$\n",
    "    \n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & 0 & \\frac{-23}{59} & \\frac{54}{59} & \\frac{-2}{59}\\\\\n",
    "    0 & 1 & \\frac{34}{59} & \\frac{51}{59} & \\frac{-15}{59}\\\\\n",
    "    0 & 0 & \\frac{129}{59} & \\frac{-426}{59} & \\frac{160}{59}\\\\\n",
    "    0 & 0 & 0 & 0 & 1\\end{bmatrix}$\n",
    "    \n",
    "$ --> \\begin{bmatrix}\n",
    "    1 & 0 & 0 & \\frac{-16}{43} & \\frac{58}{129}\\\\\n",
    "    0 & 1 & 0 & \\frac{119}{43} & \\frac{-125}{129}\\\\\n",
    "    0 & 0 & 1 & \\frac{-142}{43} & \\frac{160}{129}\\\\\n",
    "    0 & 0 & 0 & 0 & 1\\end{bmatrix}$\n",
    "\n",
    "$...$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.3 实现 Gaussian Jordan 消元法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO 实现 Gaussain Jordan 方法求解 Ax = b\n",
    "\n",
    "\"\"\" Gaussian Jordan 方法求解 Ax = b.\n",
    "    参数\n",
    "        A: 方阵 \n",
    "        b: 列向量\n",
    "        decPts: 四舍五入位数，默认为4\n",
    "        epsilon: 判读是否为0的阈值，默认 1.0e-16\n",
    "        \n",
    "    返回列向量 x 使得 Ax = b \n",
    "    返回None，如果 A，b 高度不同\n",
    "    返回None，如果 A 为奇异矩阵\n",
    "\"\"\"\n",
    "\n",
    "def gj_Solve(A, b, decPts=4, epsilon = 1.0e-16):\n",
    "    #定义一个函数，功能：求矩阵第col列中第i行到第j行的元素的绝对值的最大值,\n",
    "    #同时返回所在的行号,以及col列中所在目标行的元素的实际值\n",
    "    def maxAbs_rowidx_InColumn(M,col,i,j):\n",
    "        max_val = 0 #max value will be found\n",
    "        target_row_idx = i #index will be found\n",
    "        act_val = M[i][col] #actual value will be found\n",
    "        k = i\n",
    "        while k <= j:\n",
    "            val = abs(M[k][col])\n",
    "            if val > max_val:\n",
    "                max_val = val\n",
    "                target_row_idx = k\n",
    "                act_val = M[k][col]\n",
    "            k += 1\n",
    "        return target_row_idx, act_val, max_val\n",
    "    \n",
    "    \n",
    "    rows_A, cols_A = shape(A)\n",
    "    rows_b, cols_b = shape(b)\n",
    "    if not rows_A == rows_b:\n",
    "        return None\n",
    "    Ab = augmentMatrix(A, b)\n",
    "    for c, _ in enumerate(Ab):\n",
    "        tg_row_idx, act_val, max_val = maxAbs_rowidx_InColumn(Ab, c, c, rows_A-1)\n",
    "        if max_val < epsilon:\n",
    "            return None\n",
    "        if not tg_row_idx == c:\n",
    "            #使用第一个行变换，将绝对值最大值所在行交换到对角线元素所在行（行 c）\n",
    "            swapRows(Ab, c, tg_row_idx)\n",
    "            \n",
    "        #使用第二个行变换，将列c的对角线元素缩放为1\n",
    "        scale = 1./act_val\n",
    "        scaleRow(Ab, c, scale)\n",
    "        #多次使用第三个行变换，将列c的其他元素消为0\n",
    "        tg_rows_set = set(t for t in range(rows_A)) - set({c})\n",
    "        for r in tg_rows_set:\n",
    "            scale = -Ab[r][c]\n",
    "            if not abs(scale) < epsilon:\n",
    "                addScaledRow(Ab, r, c, scale)\n",
    "    #四舍五入Ab中每个元素\n",
    "    matxRound(Ab, decPts=4)\n",
    "     \n",
    "    #提取单位矩阵右边的矩阵或向量出来赋给result    \n",
    "    result = []\n",
    "    i = 0\n",
    "    while i < rows_A:\n",
    "        k = cols_A\n",
    "        tmp_row_list = []\n",
    "        while k < len(Ab[0]):\n",
    "            tmp_row_list.append(Ab[i][k])\n",
    "            k += 1\n",
    "        result.append(tmp_row_list)\n",
    "        i += 1\n",
    "       \n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 4.354s\n",
      "\n",
      "OK\n"
     ]
    }
   ],
   "source": [
    "# 运行以下代码测试你的 gj_Solve 函数\n",
    "%run -i -e test.py LinearRegressionTestCase.test_gj_Solve"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (选做) 2.4 算法正确判断了奇异矩阵：\n",
    "\n",
    "在算法的步骤3 中，如果发现某一列对角线和对角线以下所有元素都为0，那么则断定这个矩阵为奇异矩阵。\n",
    "\n",
    "我们用正式的语言描述这个命题，并证明为真。\n",
    "\n",
    "证明下面的命题：\n",
    "\n",
    "**如果方阵 A 可以被分为4个部分: ** \n",
    "\n",
    "$ A = \\begin{bmatrix}\n",
    "    I    & X \\\\\n",
    "    Z    & Y \\\\\n",
    "\\end{bmatrix} , \\text{其中 I 为单位矩阵，Z 为全0矩阵，Y 的第一列全0}$，\n",
    "\n",
    "**那么A为奇异矩阵。**\n",
    "\n",
    "提示：从多种角度都可以完成证明\n",
    "- 考虑矩阵 Y 和 矩阵 A 的秩\n",
    "- 考虑矩阵 Y 和 矩阵 A 的行列式\n",
    "- 考虑矩阵 A 的某一列是其他列的线性组合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO 证明："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A 矩阵的具体形状如下所示，设左上角的单位矩阵为n x n 的单位矩阵。<br />\n",
    "<br />\n",
    "$A = \\begin{bmatrix}\n",
    "1  &0  &0  &...  &0  &\\  &*  &*  &*  &...  &*\\\\\n",
    "0  &1  &0  &...  &0  &\\  &*  &*  &*  &...  &*\\\\\n",
    "0  &0  &1  &...  &0  &\\  &*  &*  &*  &...  &*\\\\\n",
    "...  &...  &...  &...  &...  &\\  &...  &...  &...  &...  &...\\\\\n",
    "0  &0  &0  &...  &1  &\\  &*  &*  &*  &...  &*\\\\  \n",
    "\\\\\n",
    "0  &0  &0  &...  &0  &\\  &0  &\\#  &\\#  &...  &\\#\\\\\n",
    "0  &0  &0  &...  &0  &\\  &0  &\\#  &\\#  &...  &\\#\\\\\n",
    "0  &0  &0  &...  &0  &\\  &0  &\\#  &\\#  &...  &\\#\\\\\n",
    "...  &...  &...  &...  &...  &\\  &...  &...  &...  &...  &...\\\\\n",
    "0  &0  &0  &...  &0  &\\  &0  &\\#  &\\#  &...  &\\#\\\\\n",
    "\\end{bmatrix}$ <br />\n",
    "<br />\n",
    "\n",
    "根据行列式计算定义的展开式，则A的行列式可以按第一列展开，<br />\n",
    "又因为第一列只有第一个元素为1，其余元素都为0，故展开式为：<br />\n",
    "$detA = a_{11}(-1)^{1+1}detA_{11} + 0 + 0 +  ... + 0，$  $\\qquad \\qquad \\qquad \\qquad \\qquad(1)$ <br />\n",
    "其中 $a_{11} = 1。$ <br />\n",
    "<br />\n",
    "$\\stackrel{\\bf .}{..} \\quad detA = detA_{11}$ <br />\n",
    "<br />\n",
    "这时$A_{11}$是A消掉第一行和第一列后的子矩阵。<br />\n",
    "<br />\n",
    "这时$\\quad$ \n",
    "$A_{11} = \\begin{bmatrix}\n",
    "I_{(n-1)(n-1)}  &X1\\\\\n",
    "Z1  &Y\\\\\n",
    "\\end{bmatrix}$ <br />\n",
    "其中， Z1依然是 0 矩阵，Y 未变。<br />\n",
    "<br />\n",
    "令$\\quad$ $A_{11} = \\begin{bmatrix}\n",
    "        I_{(n-1)(n-1)}  &X1 \\\\\n",
    "        Z1  &Y \\\\\n",
    "        \\end{bmatrix} = B$ <br />\n",
    "<br />\n",
    "则 $detA_{11} = detB = b_{11}(-1)^{1+1}detB_{11} + 0 + 0 + ... + 0,$ <br />\n",
    "道理同（1）式， $detB = detB_{11}$ <br />\n",
    "<br />\n",
    "$\\stackrel{\\bf .}{..} \\quad detA = detB = detB_{11}$ <br />\n",
    "<br />\n",
    "如此类推，展开计算到第 n 次后，<br />\n",
    "把A的左边和顶部的 n 行和 n 列都消掉了，这时就得到：<br />\n",
    "<br />\n",
    "$detA = detB = detB_{11} = ... = 1*(-1)^{1+1}detY = detY$  $\\qquad \\qquad \\qquad (2)$ <br />\n",
    "<br />\n",
    "$\\stackrel{\\bf .}{..} \\quad detA = detY$ <br />\n",
    "<br />\n",
    "而$\\quad$ $Y =\\begin{bmatrix}\n",
    "       0  &\\#  &\\#  &...  &\\#\\\\\n",
    "       0  &\\#  &\\#  &...  &\\#\\\\\n",
    "       0  &\\#  &\\#  &...  &\\#\\\\\n",
    "       ...  &...  &...  &...  &...\\\\\n",
    "       0  &\\#  &\\#  &...  &\\#\\\\\n",
    "        \\end{bmatrix}$ <br />\n",
    "\n",
    "因为 Y 的第一列全部为 0, 根据行列式计算定义的展开式，则 Y 的行列式按第一列展开得：<br />\n",
    "$detY = 0 + 0 + .... + 0 = 0$ <br />\n",
    "<br />\n",
    "$\\stackrel{\\bf .}{..} \\quad detA = detY = 0$ <br />\n",
    "<br />\n",
    "根据逆矩阵的定理，detA = 0 说明矩阵 A 不可逆，<br />\n",
    "$\\stackrel{\\bf .}{..} \\quad $ A 为奇异矩阵。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3  线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 随机生成样本点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAESCAYAAADjS5I+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGb1JREFUeJzt3X+MHGd9x/HP1xcbcnUQ8dmABbk9\nFFEgJCEoJxRIJSAQGkwERfxRtYflNpEsCFQGgdq0/oe2chsBTWqpDakJBlOvQJVCG5SaQkgjorb8\n6JmGkNSERMhnElJin1vlh0OT2N/+Mbu9vb2Z2ZnZmX1mZ98vaXXe2dnZJ6tovvs83+f5PubuAgAg\nybrQDQAA1BuBAgCQikABAEhFoAAApCJQAABSESgAAKkIFACAVAQKAEAqAgUAINVZoRtQhs2bN/vc\n3FzoZgDAWDl8+PAJd98y6LxGBIq5uTktLi6GbgYAjBUzW8pyHkNPAIBUBAoAQCoCBQAgFYECAJCK\nQAEASEWgAIAGa7eluTlp3brob7ud/xqNmB4LAFir3ZZ27pROnYqeLy1FzyVpYSH7dehRAEBD7d69\nEiS6Tp2KjudBoACAhjp2LP740lI0FCVdfFGW6xAoAKChZmeTX3OXpPUbslyHQAEADbVnjzQ9Pfx1\nCBQA0FALC9K+fVKrJZkVvw6BAgAabGFBOnpUOnMmChhFECgAYAwVWR9RdCiKdRQAMGaKro/ovrZ7\ndzQjyv25Z7N8nnmU+h5r8/Pzzn4UACbF3FwUHPq1WtEwU1Zmdtjd5wedx9ATAIyZpPURSceHRaAA\ngJoZlH9IWh+Rtm5iGAQKAKiRbv5haSlaFNfNP/QGi7ik9PR0dLwKBAoAGKFBvYUs9Zn610e0WtHz\nPIX+8iCZDQAj0j9bSYp6Ar03+XXruuU1VjOL1kKUqfbJbDM7z8zuNrMjZvaAme3qHP+kmT1qZvd2\nHttCtREAypSltzDq/EMWIYeenpf0cXd/raTLJH3YzC7ovHaTu1/SeRwK10QASJZ30VuW2Uqjzj9k\nESxQuPtj7v6Dzr+flHRE0stDtQcA8siSdO6XpbdQJP9Qxi52aWqRzDazOUlvkPS9zqGPmNl9Zrbf\nzM4N1jAASJA0jLRrV/JNO2tvobc+09Gjg4NE3oCVV/BktpltlPRtSXvc/atm9lJJJyS5pD+VtNXd\nr4l5305JOyVpdnb20qW4ZYoAUJGkpHO//mR1u71SQmN2NgoSw8xWGmaVdtZkdtBAYWbrJd0h6Rvu\nfmPM63OS7nD3C9Ouw6wnAKOWdIOOMzMjbdxYXnDoNcwsqXGY9WSSPi/pSG+QMLOtPae9T9L9o24b\nAAySpxLr8nL60NAwOYZRzJIKmaO4XNJ2SVf0TYX9lJn9yMzuk/Q2SR8L2EYAiBWXdJ6Zyfbe3imx\nw+YYRjFLKniOogwMPQGog7gFdUm6Q0NlVIItmvfIOvTEfhQAUILuzfrUKWlqSjp9OrrZP/VUNPTU\nrzs0VEYl2IWF6sp3SDWZHgsA4yIun9A7fCRFQaI7/LN3b/rQ0KZN8Z8TciV2PwIFgImWJ5GclE/Y\ntSu5NEfaArp2W3riibWfs2FD2JXY/chRAJhYWYr09cozJbar1UrOHWzeHD8sNTMjnTiR73OKqP30\nWAAILUuRvl55d5AzS57N1G7HBwlJOnky3+dUjUABYGINSiT3D0sl5RNmZtbmIczWLoTrDUJJwUiq\nV35CIlAAmGBpi9Xi8hFPPBHlD3pNT0cJ6/48RNKofjcIpfVO6pSfkAgUACZY0urqp56KT1A/95x0\nzjnxien+Qn6tVvxndoNTUpCamal2qmsRBAoAE6s7I6l/RfXycnr+IEtl10ErppNe37s3739F9QgU\nACbawkJUsC+rbk+gN3+xeXP06J1iO2hfiVHvez0MpscCmHh5S4ZL6aU60qbY1gnTYwEgo7R8Qdwv\n/rhptb3SptiOIwIFgImXli+Iy0dkWU+Rd81FnREoAEy8vPmCLOsc6rYWYhgECgBQvn2qB21aVPZ+\nEKERKAAgp/4eyMxM9Kj77KWi2I8CAAqoeg+IOqFHAQA5DbPH9TiiRwEAOfSXJu9WhZWa28OgRwFg\n4uXpIeQtTd4EBAoAjVPGrnVJ7yljj+txQ6AA0Ch5b/x5ewhppcmbikABoFHK2rUu6figqrBNRKAA\n0Ch5b/x5ewjjVPW1LAQKAI2S98ZfpIeQZxV3ExAoAFQi1FqDtF3r4towiT2EvFhHAaB0IdcadK+/\na9fqXeqWl5PbMEmrrIugRwGgdFWsNcjTQ0nata7p6x2qQo8CQOnKXmtQpIcyiesdqkKPAkDpyl5r\nUKSHMonrHaoSLFCY2XlmdreZHTGzB8xsV+f4JjO708we6vw9N1QbARRT9lqDpaV8x/O0YdIK/BUR\nskfxvKSPu/trJV0m6cNmdoGk6yXd5e6vknRX5zmAMZJnJlGWG/XUVPznJB3P2oa8q7gnlrvX4iHp\ndklXSnpQ0tbOsa2SHhz03ksvvdQB1MfBg+6tlrtZ9PfgweTzpqfdo9t09JieXnt+7+v9j2G0WvHX\nbLWGu+64kLToGe7PtchRmNmcpDdI+p6kl7r7Y5LU+fuScC0DkFeeX+lZcw+tVvxnxR3PM5REwjub\n4IHCzDZKuk3SR939iRzv22lmi2a2ePz48eoaCCCXPInnrDfqPPmGPENJJLyzCRoozGy9oiDRdvev\ndg7/wsy2dl7fKunxuPe6+z53n3f3+S1btoymwQAGyvMrPeuNOmvOI+/sqEks8FdEyFlPJunzko64\n+409L31N0o7Ov3coyl0AGBNJN/9169YOB+W5UWepr5R3KInyHdmE7FFcLmm7pCvM7N7OY5ukGyRd\naWYPKUpu3xCwjQBySqq1dPr02uGgsm/URYaSJq3AXxEWJb7H2/z8vC8uLoZuBoCOdnttraV+rVZ0\nYy77c3tXcEtR0KKXEM/MDrv7/KDzgiezATTTM8+kv17FzCKGkqpBjwJA6ebm0ldNS9X0KJAPPQoA\nwQzqLTCzaLwQKACULi15zHDQ+CFQAChd0rTXgweZWTSOCBQAYg1TVZWkcrOwcRGANcrYypTtRZuD\nHgWANarYyhTji0ABYA2qqqIXgQKoiTrttEZVVfQiUAA1ULed1vIU66tTgEM1CBRADdQtJ5B11lLd\nAhyqQQkPoAbWrYtutP3MoqqmdZVUqoPyHOOBEh7AGBlVTiBpmKjo8BFJ78lAoABqYBQ7rSUNE113\nXfHhI5Lek4FAAdTAKFYyJ+VBPvvZ+OO7dg2+JluJTgYCBVATVe+0lnc4aHl5cK+CUh2TgWQ2MCGy\n7BHRb2ZGOnGikuagBkhmA1glaS/rNFl6FWg+AgUwIbrDRFNT+d5HfScQKIAJsrAgHTiwtmexYUPy\ne5jqCgIFMKaKrn2IS0Dv3x/lI+Iw1RUECmAMZSmdkRZI4mZY7d3LVFfEI1AANZV2ox9UG6pIDSam\nuiIJ02OBGurfYU6Kft13b9yDakNRgwlZMD0WGGODegyDSmdQgwllIlAANZR0Q19akjZvjl8E15tP\noAYTykSgAGoo7Ya+vCw9/fTqYzMzq/MJ1GBCmQgUQA30J663bcu3inrjxtVJZxLTKBPJbCCwpMT1\njh3SoUPZ6jPVfYMj1FPtk9lmtt/MHjez+3uOfdLMHjWzezuPbaHaB2RRxn7RSYnrQ4eiGUqt1uBr\nkHtAlUIOPX1R0lUxx29y90s6j0MjbhOQWVn7RQ+aoTSomB+5B1QtWKBw93sknQz1+cCwBk1hzWrQ\nDKX+fMPMTPQg94BRqWMy+yNmdl9naOrc0I0BkpS1ViHLDKXekhsnTkSPqjY4AvrVLVB8VtL5ki6R\n9Jikv0g60cx2mtmimS0eP358VO0D/l9ZaxWYoYS6yxUozOwnZvYHZvayKhrj7r9w99PufkbS5yS9\nMeXcfe4+7+7zW7ZsqaI5QKoy1yr09hj27ImGr4ZJkANlytujeE7Sn0s6Zmb/YGZXm1lpvRIz29rz\n9H2S7k86Fwitip5AWQlyoEy5bvLu/jpJb5Z0QNLbJN0u6WdmtsfMzs9zLTP7sqTvSHq1mT1iZtdK\n+pSZ/cjM7utc/2N5rglUqd2OymeYRY/Nm6Pj/eW6h1FWghwoU+EFd2Y2Lek3JV2rKHi4pG9LulXS\nbe7+v2U1chAW3KFq7bZ0zTXSs8+uPr5unfSlL5WXTxhUFRYoU+UL7tz9lLt/wd1/TdJrJH1F0lsl\n/a2kn5vZTWbGMiA0wu7da4OEFN28d+0q73Mo5oc6Giq/YGZTZvY+STcq6l24pLslfVfS70k6Ymbv\nHbqVQGBpU16Xl8v7HIr5oY4KBQoze42ZfVrSo5JukzQv6TOSftXd3+Hu71bUy3hQ0qfKaiwQyqh+\n0TNVFnWUK0dhZtcoyklc1jn0LUn7JN3u7s/HnP87km5197OGb2oychSoWrstfeAD8a/NzMTvDwHU\nXVU5ilslvVLSDZLOd/dfd/fb4oJEx38qylkAY21hQfrQh9Ye37BB2ru3+HXLKCoIVC1voHi/pPPc\nfbe7Hx10srt/391/t1DLgJq5+Wbp4MHVw0L79xcfFmLNBMYF+1EAgczNxe810WpFazKAqtV+Pwqg\nCYYZOiqrqCBQNQIFUNCwQ0esmcC4IFCglsYhyTtsuQ3WTGBcEChQO3l/qVcRVLJcc9ihI9ZMYFyQ\nzEbt5EnydoNK7y/76enhbrhZr0kyGuOOZDbGVp5f6lVUW816TYaOMCkIFKidPEnepKCytFR8GCpr\noGLoCJOCQIHayfNLPW2GUNEFbHkCVe/OdOxfjaYiUKB28vxSjwsqvYoMQzGkBKxGoEAtDfql3p2V\ntH27dPbZUWG+JHkXsDGkBKxWaVVXoAr9s5KWl6Nf/DMz8XtDFFnAtrBAYAC66FFg7CTNSpIYMgKq\nQKDA2EkaSjp5kiEjoAoMPWHszM7GL3SbnWXICKgCPQoEVaT8Rp5ZSeNQMwqoO3oUCKY/Kd1d9yCl\n9wq6r+3eHQ1Dzc5GQSJuZlSR6wNYjVpPCKbqWknUYgLSUesJtVf1xj1sDASUg0CBYKreuIeNgYBy\nECgQRLstnTgR/9q2bfHnV5n0BpCMQIGR6yaZn346/vVDh+LPz7vlKKU4gHKQzMbIJSWZu8yiGk+D\nzicpDQyHZDZqa1AyuT+HkBRU0oINgPIECxRmtt/MHjez+3uObTKzO83soc7fc0O1D9VJSybH5RCm\npuLP7eYrWEwHVCtkj+KLkq7qO3a9pLvc/VWS7uo8R80Mu9o5aQ+JmZn4HMLp0/HXOXMmf94CQH7B\nAoW73yPpZN/h90o60Pn3AUm/MdJGYaCiieVecUnmgwejWVBxieZWK9t1h90rG0C8oMlsM5uTdIe7\nX9h5/j/u/uKe1//b3QcOP5HMHp0QieX+Uhxp+hPhAJI1PpltZjvNbNHMFo8fPx66OROj6tXOccNa\ncT2QpB3tWEwHlK9ugeIXZrZVkjp/H0860d33ufu8u89v2bJlZA2cdFWudk4b1urfGnXvXhbTAaNS\nt0DxNUk7Ov/eIen2gG1BjLhEtFn8auq8knaui8s7sJgOGJ2Q02O/LOk7kl5tZo+Y2bWSbpB0pZk9\nJOnKznPUyMKCtGNHdHPucpcOHBh+xlHasFbSkFRvL4MgAVSDldnIraqEdtJ1Z2akZ55Z3duYnqYH\nAQyr8clshFNVQjupiJ+UfUgKQPkIFFhj0IK6TZvi35d0PKukvMPJ/tU2HewrAYwGgQKrlLGgbhhx\neQf2lQDCIlBglSwzj5J+4ScdHxb7SgBhESiwSpb8w6h/4TMVFgiLQDFBshTzyxIEQvzCZyosEA6B\nYkJkzT1kCQL8wgcmC+soJkSetQ/tdpSTOHYs6kns2UMQAJqIdRQNVXQviDxrHxjmAdCLQDFG4oaP\ntm+Phn8GBQ2mmAIoikAxRuKmrnZHDgetd2CKKYCiCBRjZNBK5LSyFiSgARR1VugGILvZ2fiEdK+0\n1xcWCAwA8qNHMUbiho/6mY2u3AaAyUCgGDNnn53+ujtVVQGUi0AxJroznpaXB5/bO/xUdDotAHSR\noxgTcTOekkxNRX+7waX7vu7MKIlcBYDs6FGMiTx7L5w+Hf3Nswc1ACQhUIyJPAvjWq3ob9IMqEEz\npwCgF4FiTMTNeFq/XtqwYfWx3kV03SGofknHASAOgaKgvEnipPOzXiduwdwXviDt35+8iK47BNUv\n6TgAxKF6bAH9SWIp+iWftNI56fwdO6QDB7JfJ688FWMBTB6qx1Yob5I46fx9+6pNNuet78RUWgBx\nCBQF5CnZnXY8aQgozwynNHnqO2Xd2AjA5GHoqYC8QzpJ5ycJMTTEMBUweRh6qlDeIZ0sNZqyXKdK\neXtDACYHgaKAvCW7e89PE7L0NxsbAUhCoCgo73ah3fPN4l83y3adqhLObGwEIAmBIocybtLD/HKv\nMuHMxkYAkhAoMirrJp3ll3tSQKq6dlPeXhKAyVDLWU9mdlTSk5JOS3p+UFZ+FLOeypwV1G5HN/dj\nx6KexJ49KzfltMV827ev7JHdyyy6uQNAHllnPdU5UMy7+4ks548iUKxbN5qbdFpAkpjCCqA8TI8t\n2ahmBaVNUyXhDCCEugYKl/RNMztsZjtDN0Yq7yY9KCG+aVP8+2ZnSTgDCKOuO9xd7u4/N7OXSLrT\nzH7s7vf0ntAJIDslaXYEk/27N+Ok3EIWg3aca7elJ59c+77161cC0sICgQHAaNUyR9HLzD4p6Sl3\n/0zSOaMu4VHUoIR40uszM9KJTNkaAMhubHMUZvYrZnZO99+S3inp/qo/dxSVUweVyUh6/eTJ8tsC\nAFnVLlBIeqmkfzGzH0r6vqR/dPd/qvID49ZIbN8uXXdduZ8zKCFOGQ0AdVS7QOHuP3X313cer3P3\nyuf0xC1kc5duuaXcnsWghDizmgDUUe0CRQhJQz7u5a16lgbPWmJWE4A6qn0yO4thk9lp+0Ww6hlA\nU41tMjuEPXuSq7oOyg+wfSiApiNQKBraueKK+Ne2bUt+H9uHApgEExko4noBDz8cf+6hQ8nXqbqa\nKwDUQV1XZlcmaXV0/w2/K20rULYPBTAJJq5HkdQLmJqKP392NjkPMWjdA/kLAE3QiFlPL3jBvD/3\n3GKm+ktJ5cKlaM1C/z4QO3ZIBw7E7w8hJe8dkfYa010B1MFEzXp69tnsyeSkXkB3zUL/GoZDh5Lz\nEGnrHshfAGiKRvQozOZdWllHkbaRT9oOcnG/9ItuWDSqjY4AoKiJ6lH0S0sm5139XLT+EnWbADRF\nIwPFoJvxwkLU4zhzJvqbljMoWn+Juk0AmqJxgaLsm3HR+kvUbQLQFI3IUeSZ9QQAiGTNUTRiwd1F\nF0ljsMEdAIylxg09AQDK1chAwYpoAChP4wJFUkXX664jeABAEY3IUfRKWhF9yy0rC+C6wUMi8Q0A\ngzSuR5G2rWkvymkAQDaNCxR5Vj5TDhwABmtcoIhbEV10m1MAQAMDRdyK6A9+kHIaAFBU4wKFtLaW\n0803U04DAIpq3KynJAsLBAYAKKKRPQoAQHkIFACAVAQKAEAqAgUAIBWBAgCQqhEbF5nZcUlLgZux\nWdKJwG2oC76LFXwXK/guVtTlu2i5+5ZBJzUiUNSBmS1m2SlqEvBdrOC7WMF3sWLcvguGngAAqQgU\nAIBUBIry7AvdgBrhu1jBd7GC72LFWH0X5CgAAKnoUQAAUhEoKmBmnzAzN7PNodsSipl92sx+bGb3\nmdnfm9mLQ7dp1MzsKjN70MweNrPrQ7cnFDM7z8zuNrMjZvaAme0K3abQzGzKzP7DzO4I3ZYsCBQl\nM7PzJF0padL3z7tT0oXufrGkn0j6w8DtGSkzm5L015LeJekCSb9lZheEbVUwz0v6uLu/VtJlkj48\nwd9F1y5JR0I3IisCRflukvT7kiY6+ePu33T35ztPvyvpFSHbE8AbJT3s7j9192clfUXSewO3KQh3\nf8zdf9D595OKbpAvD9uqcMzsFZLeLenW0G3JikBRIjN7j6RH3f2HodtSM9dI+nroRozYyyX9rOf5\nI5rgm2OXmc1JeoOk74VtSVB/qejH5JnQDclqYjYuKouZfUvSy2Je2i3pjyS9c7QtCiftu3D32zvn\n7FY09NAeZdtqIG6n9onuZZrZRkm3Sfqouz8Ruj0hmNnVkh5398Nm9tbQ7cmKQJGTu78j7riZXSTp\nlZJ+aGZSNNTyAzN7o7v/1wibODJJ30WXme2QdLWkt/vkzcN+RNJ5Pc9fIenngdoSnJmtVxQk2u7+\n1dDtCehySe8xs22SXijpRWZ20N0/ELhdqVhHUREzOypp3t3rUPhr5MzsKkk3SnqLux8P3Z5RM7Oz\nFCXx3y7pUUn/Lum33f2BoA0LwKJfTgcknXT3j4ZuT110ehSfcPerQ7dlEHIUqMpfSTpH0p1mdq+Z\n3RK6QaPUSeR/RNI3FCVv/24Sg0TH5ZK2S7qi8//CvZ1f1BgT9CgAAKnoUQAAUhEoAACpCBQAgFQE\nCgBAKgIFACAVgQIAkIpAAQBIRaAAAKQiUAAAUhEogBKZ2Vlm9q9m9pSZvabvtZ2dnQ//JFT7gCIo\n4QGUzMxaku6VtCTpMnf/pZm9TlFhwMOS3urup0O2EciDHgVQMndfknStpNdL+oyZna1oh7tfSlog\nSGDc0KMAKmJmN0v6kKR/k/RmSe+f8L0YMKYIFEBFzOyFku6XdL6kz7n7zsBNAgph6AmozsWSZjv/\nvrCzmREwdggUQAXM7EWK8hInFO2n/iZJfxy0UUBB/MIBqvE3klqSrnT3fzazSyRdb2bfcve7A7cN\nyIUcBVAyM7tW0q2S/szdd3eOvVjRlNn1ki529+WATQRyIVAAJeossjusKCi8pbN3dve1N0m6R9LX\n3f09gZoI5EagAACkIpkNAEhFoAAApCJQAABSESgAAKkIFACAVAQKAEAqAgUAIBWBAgCQikABAEhF\noAAApPo/CFx5T0PToBUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b42d10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 不要修改这里！\n",
    "# 运行一次就够了！\n",
    "from helper import *\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "X,Y = generatePoints(num=100)\n",
    "\n",
    "## 可视化\n",
    "plt.xlim((-5,5))\n",
    "plt.xlabel('x',fontsize=18)\n",
    "plt.ylabel('y',fontsize=18)\n",
    "plt.scatter(X,Y,c='b')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 拟合一条直线\n",
    "\n",
    "### 3.2.1 猜测一条直线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAESCAYAAADjS5I+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VNX5x/HPw04UFwJWCiZYgyhU\nXIiKa0Vxr3vdGpcKrT/RKiqICIqAoqDiWq2ioChR6y7Wpa513wKCshMrUEHZLSAgkDy/P24CWWYm\nM8kkdzL5vl+veWUyuXPnMC+9zz3nOec55u6IiIhE0yjsBoiISGpToBARkZgUKEREJCYFChERiUmB\nQkREYlKgEBGRmBQoREQkJgUKERGJSYFCRERiahJ2A5KhTZs23rFjx7CbISJSr0yePHm5u7et6ri0\nCBQdO3akoKAg7GaIiNQrZrYgnuM09CQiIjEpUIiISEwKFCIiEpMChYiIxKRAISIiMSlQiIhITAoU\nIiISkwKFiIjEpEAhIpLG8vOhY0do1Cj4mZ+f+DnSYmW2iIhUlp8PF18M69YFvy9YEPwOkJcX/3nM\n3ZPfujqWm5vrKuEhIlJex45BcKiocWMoLgb3vTe6T2te1XnUoxARSVMLF0Z+vaio9FnTZvGcRzkK\nEZE0lZWVnPMoUIiIpKmRIyEjI/LfjuLtuM+jQCEikqby8mDsWMjOBrMgN9GV6bzKCbzN0XGfR4FC\nRCSN5eXB/PlQvPhHZh1+MdPYm4P4lP7cEfc5FChEROqhuNdHrFsHN90EOTl0+ugx5h53BT07FHKX\n9Qc2bYznszTrSUSknolrfURRETzxBAwZAosXwxlnwKhR7JmTw9SSQ8y+/iaez1OPQkSknhkyZGuQ\nKLVuXfA6AO+8A7m5cNFF0KEDfPghPPcc5ORU6/MUKEREUkxVw0rR1kdsu2AGnHgi9OoFq1bBU0/B\nZ5/BoYfWqD0KFCIiKaR0WGnBAnDfOqxUNlhUXB+xE0v4O5cwjW7w8cdw++0wezacc04w3amGFChE\nROpQVb2FKoeV2Lo+oiXrGMxICsmhD+OYd8xfobAQBgyAFi2S1mYls0VE6kg8Sehow0plX887t5hd\nP5xIx0eG8Oui73mj5WlsvGk0J/fvVCvtVo9CRKSOxNNbiFZ2Y8vr774Lubkc/NCF/Hq/dvDBBxy3\n7oVaCxIQYqAws13M7D0zm2VmM8ysX8nrw8xskZlNLXmcEFYbRURiSXSvh3h6C5HKbmRkwH2XzoKT\nToKjjoKVK+HJJ4NE9WGH1eSfEJcwexSbgf7uvifQA7jMzLqU/O0ud9+n5PFaeE0UEYksnqRzRVX2\nFqhcdmO/Dkv56qBLOWnwXvDBBzB6dJCoPvfcIEKRnM2JYgktULj7D+4+peT5GmAW0D6s9oiIJCLa\nMFK/ftEv2tF6CyNHln8tLw/mz1pP8chbmfy/HHZ//2Ho2xe+/RYGDiyXqK5OwEqYu4f+ADoCC4Ht\ngGHAfOBrYDywY5T3XAwUAAVZWVkuIlKXzNyDS3PsR0aG+8SJW983caJ7dnbw/uzs8n9zd/eiIvfH\nH3ffZZfgBKec4j57dtR2ZGdH/tzs7Kr/DUCBx3GNDn2HOzPbFngfGOnuL5jZr4DlgAM3Ae3cvXes\nc2iHOxGpa9F2j4skMxO23TbIRWRlBT2IiFuR/vvf0L8/TJkC3bvDmDHwu9/FPHejRkFoqMgs2MUu\nFjOb7O65VbU/1FlPZtYUeB7Id/cXANx9ibsXuXsx8DBwQJhtFBGJJNZeDxWtWBF7aOiV22fzVsbJ\n0LMn309bzsd9J8IXX1QZJCC+vEdNhTnryYBxwCx3v7PM6+3KHHYaML2u2yYiUpWKSefs7KDnEI8t\nU2KXLWPu0Zdx/MDfcsD697mWUXQqms0xE/LIfyq+y3O8eY8aiWd8qjYewKEEw0tfA1NLHicATwDf\nlLw+iWDoKea5unfvXvVgnIhILZs4MchJVJW3aME6H8St7q1a+SYa+31c5m1YmnCOoeznxsx7REF9\nyVEkg3IUIhK2/Pygl7BgQbCTXFFR0MtYuzYYegIwijmXp7iFwWSzEE4+mT0njWY2e1Q6Xzw5hpqq\nFzkKEZH6JtKahbJTVCEIEqXDP/fcEzw/nPf5ggPI5zxWWhveHvwuvPwyyzIrBwlIbo6hphQoRKRB\nS2SxWrQ1C/36RS/NkZc7h9l7nsr7HMFOLOXKzCeYOeFLeo3sSX4+rF5d+XOaNUtyjqGGNPQkIg1W\nxSJ9ENz9jx0befpqIlNi27CMoYzgEh5kPS15cIfryLrzSs65qOXWY9psHZYqKzMTli9P7N9SHRp6\nEhGpQjxF+sqKVquprOZs4Bpuo5Ac+vJ3HuYv5FDItT9dR5+/ttzSY8nPjxwkICjllEoUKESkwaqq\nSF/FYanWrSMfn5kJ27Qs5lyeZA6duY1r+YDD2YtvuIwHWMZOQPkgFC0YQWrlJ0CBQkQasFiL1SLl\nI1avDvIHZWVkwJN9P2R+ux48SR4ryOTcnd7hZF5hNntWOndpEIrVO0ml/AQoUIhIAxZtdfXatZET\n1Js2QatWWxfZHfHruczqcjrH3Hw4bX5ZDBMmsF9RAU8tOZLs7MifWRqcogWpzMwo5T1CpEAhIg1W\n6erqiiuqV6yInT+YX7Cc4sv78d7SrmTNfgtuvhnmzoULLthS+ruqFdPR/n7PPUn4hyWZAoWINGh5\neUHBvng0ZwM3b3875ORQfN/fyG/Rh53XFtLmriG0ycooN8U2UomPsrOpqvp7KtH0WBFp8KJVYN3K\nOZt/MMquo6PPZ9HeJ3DynNuZsqFLxKNjTbFNJZoeKyISp1j5gjN+9RGf0YOnOZftdtke3nqLQ356\nNWqQgNhTbOsjBQoRafAi5Qt+26KQKb85g+eWHMaB7RfBY4/R+j+ToVevuNZTxHNMfaFAISINXtl8\nQSYrGNfqSqZu6kLWzH/BTTcFieoLLwyq/RHfOodUWwtREwoUIiJA3h9+Yf7lY1i+Qw69f76Pxn0u\ngsJCuP76St2NqjYtSvp+ECFToBCRhs0dnnkG9twTBgyAgw6CadPgoYdg550jvqXijKXMzOCR6rOX\nqqtJ2A0QEQnNJ58Ee1R/9hl06wZvvglHH13l20r3nqhyD+w0oR6FiDQ8334LZ54JhxwS1OYYPx6m\nTIk7SEQqNR6rPHl9p0AhIg3HypVw9dXBMNPrr8Pw4TBvHvnNLqLjbo3j2pMi0Yqz6UBDTyKS/n75\nBe6/P5jBtHo19O4NI0ZAu3aV9qQo7SFA5OGkqirOpiP1KEQk7WwpD27OpW2fZU1WlyAX0aMHTJ0K\nDz8M7doBifcQYlWcTVcKFCKSVkp7CO0WfMpHHMIDy89iwbJteOfafwXDTXvtVe74RHsIVRX7S0cK\nFCKSVh4c+B/GrzubTzmYXfmOPjzC3v4VfZ4+JuLxifYQ6lMxv2RRjkJE0sOqVXDzzbyz+D420ZRh\n3MgdDOBngtKwsXoIkfbNjtVDyMtL78BQkXoUIlK/bdwId98Nu+0Gd93Fi9ucTyfmMZxhW4IEBBVi\nI81maog9hEQpUIhIrai433TS1xm4w/PPQ5cucNVVkJsLU6ey+aFx/C/j15UOLyqKvt4hLw/mz4fi\n4uCngkR5ChQiknS1vijt88/hsMPgD3+AFi2CJPWbb0K3blt6CCX1+8pJ9/UOtUWBQkSSrjYWpeXn\nw6Htv+NpOwd69GD99G+Daa5Tp8Jxx5U7Ni8v6B1Eks7rHWqLAoWIJF2yF6U9O3YVy/50De8s3oOT\nmcRwhpK9cR75Lf8MTSLPyWmI6x1qiwKFiCRd0i7SGzfCvffSq28OV2weQz55dGIewxjOsvXbxuyh\nxLveodZzKWkgtEBhZruY2XtmNsvMZphZv5LXW5vZW2Y2r+TnjmG1UUSqJ5FFaREv1O7w4ovQtSv0\n60dB8X7sy1f0YTyLab/lvQsWRG9DPLOZGmKBv2px91AeQDtgv5LnrYC5QBfgNmBQyeuDgNFVnat7\n9+4uIqlj4kT3zEz34PIbPJ84MfJxGRlbjwP3w5p/7kt2PzT4pUsX99de88aNissdU/po3Lhm7czO\nrnxOCF5vCIACj+N6HVqPwt1/cPcpJc/XALOA9sApwISSwyYAp4bTQhGpjtK79BUrtr62fn3kY8sm\nvbOZz5Ocywe/HIgVzg02Dpo2DY4/nqJii/j+oqLInx/vUFJDLPBXLfFEk9p+AB2BhcB2wE8V/rYq\nynsuBgqAgqysrKRGWRGpvkTu0s3ct2eVj+Ya30Az/5mWPoLrvRWrq3XOSD2UjIzIvZlE25qOSPUe\nRSkz2xZ4HrjS3VfH+z53H+vuue6e27Zt29proIgkJO679E2buGGH+ygkhwHcwVOcy+7MZSg30Tq7\nVblD4815JDottyEW+KuOUAOFmTUlCBL57v5CyctLzKxdyd/bAUvDap+IJK7KGU/u8NJL0LUrw1dd\nwfRGe9OdyVzEYyyiQ8QLdbxlNhIdSlL5jviEOevJgHHALHe/s8yfJgEXljy/EHi5rtsmItUX6S69\nWTNYuxYOsC/5vOURcNppwfqHf/6TRRPeZlX2vlVeqOMps1Gdabkq31G1MHsUhwDnA0ea2dSSxwnA\nKOBoM5sHHF3yu4jUE6V36ZmZW1/beeMC7lmRxxccwK6/zOKKpn/nyUFfw4knkneeJe1CraGk2hFa\nmXF3/wiIPJUBjqrLtohI8q1fD9vxP67jVq7kbhxjJIMZzbWs2bQdk4bCHy9I7meWBpkhQ4Lhpqys\nIEiol1AzFiS+67fc3FwvKCgIuxkiUiInexPHLhzLMIbRluVM4AKu52a+Z5ctx5hFr8ckdcPMJrt7\nblXHaeMiEUked3jlFf65cCB7MIf3OIL+jOEr9qt0qGou1R+hT48VkTRRUAA9e8Ipp9C0CZzEJI7k\n3YhBQnmD+kWBQkRqZuFCOP982H9/mDkTHniAzx/5hnczTqJsGtJKnmoKav2jQCEiEVVZCmP1ahg8\nGDp3hueeg+uug3nzoG9f/nhh00rrE554IhiZ0hTU+kc5ChGppLReU+kq59KqqgB5Z20KNgwaNgyW\nLYPzzgvGkSokHfLyFBDShXoUIlJJ5FIYzrtXvQLdusFllwV7VRcUBF0FZabTmgKFSIpIpQ10Kpa8\n2JcpvMNRjFt2cjCn9eWX4b33oHv3cBoodUqBQiQFpNoGOqUdhA78lwlcwBS6sxffcEPrv8H06XDy\nyVuz05L2FChEUkCiVU9r2+jr1zC6yRDmsjtn8QyjuJZuLQvZ497LoGnTcsemUk9IaoeS2SIpIGU2\n0Nm8GR55hLNvvBE2L+XFbfK46ueRkJ3NHRFKYcRMeiuRnTbUoxBJAdWpelod0e7+8yc6F+30KjOb\ndoO+fVm6Y2f44gtOWzuR+Z4ddUprqvWEpHYoUIikgLqoehotD3LLWVNpf2EvHl32e5qwmVN5kV0X\nvk/+3P2rPGfK9ISkVilQiKSAuthAp+Ldf3u+5/51f2LQs/vx2+JpXM69dGUGL3Mq69Yb/fpVfc66\n6glJuBQoRFJEbW+gU3qXvy1rGMENzGV3zuUpbucacijkb1zOZrYmqlesqDoxrf0fGgYFCpEGYtdd\nNnMxD1FIDjdwMy9xKp2ZwyBG8z92iPieqnoV2kq0YdCsJ5F05w6vv87komvYgZl8yKGczCS+4MAq\n31raq4h14VepjvSnHoVIOps2DY45Bk48kR1abuSDfs/Ts9EHcQWJUprBJAoUIulo0SLo3Rv23Rem\nTIF77oEZMzj87tOZ8LhVyis0axb9VJrBJAoUIvVUxDURa9fCjTfC7rsHL/TvD99+C1dcsSUaRMor\njB8PmZmRP0czmESBQqQeqrgm4r8Livik98Os75ADI0bASSfx0qjZdHz2dhq13qFSaY1IM6zuuUcz\nmCQyBQqReqjsmohjeYOp7MP9Gy9mxobd4NNPyT/pafKu3zWhIoOawSTRKFCIpKhYxfYWLoS9+Jo3\nOJY3OJ6WrOcMnuOAXz6CHj2qXVqjttdySP2k6bEiKShmsb2ei3kq4wbO/PlRfmIHruQuHuBSNtGM\n7OzgGJXWkGRSoBBJQZF6BLZuLfPOu4OfuZ3T2MRdXMXNXM9P7AiUzydkZQXBpSIlpqU6NPQkkoLK\n3vk3oojejGMuuzOM4fyT37MHsxnAmC1BIjOzfD5BpTUkmRQoRFJQ6Z3/0bzJV+zLOP7MfDpyEJ9w\nDv/gO35T7vhtty2fT1BiWpJJgUIkBVRMXPc5cDpvNjqONzmWbVnLmTzDIXzMZxwU8f2Rcg9KTEuy\nhBYozGy8mS01s+llXhtmZovMbGrJ44Sw2icSj2RsA1p2TcSv/AeGLPgLg5/Zm4ObfsFNO97Jnszi\nOc4Eou9RrdyD1KYwexSPAcdFeP0ud9+n5PFaHbdJJG7RNgJKNFgMGQKs+5mhDGcenbiQCdxDPw5u\nW8gNK69i/MTmlfINZSn3ILUttEDh7h8AK8P6fJGaSso2oEVFHLVgPPPoxHCG8TrH04WZ9OdOvlnU\nGqicb8jMDB7KPUhdScXpsX81swuAAqC/u68Ku0EikdR4rcJbb8GAAYzjaz7jQM7kWT7hkC1/Ljuc\npFLeEqZUS2b/HdgN2Af4ARgT7UAzu9jMCsysYNmyZXXVPpEtqr0N6PTpcMIJQfnvNWv48K//4KiW\nn5YLEhpOklSSUKAws7lmdq2Z7VwbjXH3Je5e5O7FwMPAATGOHevuue6e27Zt29pojkhMCa9V+PHH\nIImx997w6adwxx0waxaH3XcWYx+2ckNLLVvC+edXP0EukkyJ9ig2AbcCC83sJTP7vZklrVdiZu3K\n/HoaMD3asSJhi3utwrp1cNNNkJMDjz4Kl18OhYVBCfDmzbeca/58eOIJWL8+2FmuJglykWRK6CLv\n7l2Bg4EJQE/gZeC/ZjbSzHZL5Fxm9hTwKdDZzL43sz7AbWb2jZl9XXL+qxI5p0htys+HNm2CoGAW\nPIcYaxWKiuCxx6BTJxg6FI49FmbOhLvvjrr5Q1IS5CJJZu5evTeaZQBnA30IgocD7wOPAM+7+y/J\namRVcnNzvaCgoK4+Thqg/Pxgw7iNG8u/3qgRPP54hF7EO+/AgAEwdSoccACMGQOHHlrl5zRqFPQk\nKjILgpFIMpnZZHfPreq4ag8bufs6d3/U3Q8F9gCeBo4AngAWm9ldZqZlQJIWhgypHCQguHj361fm\nhRkz4MQToVcvWLUKnnoqyEfEESSgBglykVpUo/yCmTU2s9OAOwl6Fw68B3wGXA7MMrNTatxKkZDF\nmvK6YgWwZAlccgl06wYffwy33QazZ8M55wTdhDipmJ+komoFCjPbw8xuBxYBzwO5wB3A7u7ey91P\nJOhlzAFuS1ZjRcIS7Y6+JesYzMggUT1uHFx2WZCovuYaaNEi4c9RMT9JRYlOj+1tZh8DM4CrgWnA\nmUAHdx/k7t+WHuvuhcC9BOsiROq1inf0RjHn8zhz6MxIroejjw6Gne69d2uWOw6RakWpmJ+kmkR7\nFI8AuwKjgN3c/Vh3f97dN0c5fiZBzkKkXsvLg759g+c9eZcCcnmcC1liO/PW9e/DCy/A7rsndM5k\n1YoSqW0JzXoqyUdMcvei2mtS4jTrSerErFl8nzeQDl/9kwVkMabNrRx45znknV+9VF/HjpF3ocvO\nDnoSIrUt3llPCdV6cvcXq98kkXpq6VIYNgzGjqXDNtvA6NFkX3EF91YjB1GW9rWW+iLVaj2JpI71\n6+HWW4NE9dixwdhTYSEMHLglUV2T/Sg0FVbqCwUKkYqKi4NaGp07w+DBcOSRQaL6vvugTF2xmuYY\nNBVW6gsFCklJydg5rlr+/W/Yf3+44AK+XrITR/BvOk59ifyCzpUOrWm5DU2FlXrD3ev9o3v37i7p\nY+JE94wM9+A+PXhkZASvRzs+O9vdLPgZ7biYZs1yP+kkd/C1mbv4Rc0mulEU8/PNyrex9GFWjc8X\nCQFQ4HFcY9WjkJSTyJ16jaeYLlsWLJL77W+D3sStt7Jfxhwe3ZhH2f89In2+cgzSUChQSMpJZDZQ\ntYd/1q+HUaNgt93goYfg//4vSFQPGsS871vG9fnKMUhDoUAhKSeRO/VoQWXBgii5jeLi4MU99oDr\nroMjjgh2nLv/fthpp4Q+XzkGaSgUKCTlJHKnHmuYp9Iw1PvvByW/zzsvKLPx7rswaVIQNKr5+Sq3\nIQ2BAoWknHju1EtnRS1YEBwTzbp1MG7gHDj11KD3sGRJsIHEl19Cz57V/nyRhqTaGxelEpXwaFhK\nE9hlcxNmlTf8acMyhjKCS3iQptu2CIaarroq2JBaRGqnhIdIKoiUwHaHxo2D3Uebs4EruJchjGQb\nfubpbf/CeYXD4Fe/CqW9IvWdhp6k3omWwC4uKuZPzZ5kDp25jWv5gMM5sMXX2IN/V5AQqQEFCql3\nIiWwD+VDpjTrwaMb81jbtDW9eJvLs1/h6ke6KLcgUkMKFBKq6pTqKDsrqRNzeZ7T+ZDD2X2bxTBh\nAl03TOZtP2pLqe5QSoGIpBHlKCQ0FZPSpdNZIfYMo7w8aL5mOWsG3sR5ax7gF2vB1D/czD6PXVVu\nXmt1zy8i5WnWk4SmWhv3bNgQVHEdORLWrIG//CXYK2LnnZNzfpEGRLOeJOUltHGPO/zjH8EU1/nz\n4YQT4LbboGvX5JxfRKJSjkJCE3epjo8+gh494NxzYfvt4a234NVXYwaJhM4vIjEpUEgo8vNh+fLI\nfzvhhJInhYVwxhlw2GGsm/c9AzIfpcm0yXT8c6+Ek96lVLRPJHEaepI6F2lldVmfvLICmt0EDzwA\nzZox7YwR9Hrtapav3wZILOkNwQK9hQuDnsTIkUpkiyRKyWypc9GSzM34hb/yN67nZnZstBr69IER\nI+jYY2clpUVqQbzJbA09SZ2rnEx2zuQZZrEnYxjA1BYHwbRpQSW+nSMHCYgcbEQk+UILFGY23syW\nmtn0Mq+1NrO3zGxeyc8dw2qf1J6yyeSD+IRPOJhnOJs1tOKk5m+y+JHXgh3nSjRuHPk8pYvotJhO\npHaF2aN4DDiuwmuDgHfcvRPwTsnvkmKqs5q6rJEjoWuLb3mGM/mEQ8hmAb0Zx9Gtp3DOuKMr5RCK\niiKfp7i4BlugikjcQgsU7v4BsLLCy6cAE0qeTwBOrdNGSZVqvEf1ypXkTb6aaZv25ER7jWEM46hd\n5nHUxN4sXdE4YqI5Ozu+U8e1BaqIJCzVchS/cvcfAEp+7hRye6SCau9R/csvcOedwR7Vd99N4z9d\nQMaiQob5jcxauE3MmUiRprlGo8V0IsmXaoEibmZ2sZkVmFnBsmXLwm5Og5Hwamd3ePZZ6NIF+veH\nAw+EqVPhkUegXbtKh0ca1oq041xmZuSP02I6keRLtUCxxMzaAZT8XBrtQHcf6+657p7btm3bOmtg\nQ5fQaudPP4VDDoGzzoJttoE33gge3bpFPEesYa2Ke1Pfc48W04nUlVQLFJOAC0ueXwi8HGJbJIJI\nw0BmZVZTA/znP3D22XDwwfDdd0Hv4auv4NhjY547kWEt7WstUofcPZQH8BTwA7AJ+B7oA2QSzHaa\nV/KzdTzn6t69u0vd6dvX3cw9uO8PHhkZ7s88tNL96qvdmzYNXrjxRvc1a+I+b8Vzlj7M3CdOdM/O\nDp5nZwe/i0jNAAUexzVWK7MlYRVXVjdlI5fyAMMajWAH/wkuughGjID27Wt03lKZmbB+ffneRkaG\nehAiNaWV2VJrtiaundN5npl04W6u4ovi3CBRPW5cwkECohfxg2rOtBKRpFCgkEqqWlDXujUcwOd8\nyGE8zx/YQAuO43X+2PpfURPV8YiWd1hZcbVNCU2FFakbChRSTpUL6r77jofXnMPn9CCHQv7CWPZh\nKv/iuODqXkMVZzfl5WlfCZGwKVBIOdFmHo0etAquuQb22INjN05iBDfQiXk8wl8oKqlWH+3Ov6a0\nr4RIuBQopJyKwzlN2cjl3Mt73+fAmDGQl0fP9vO4kRGspVW5Y2vrDl9TYUXCpUDRgMRTzG/rxd45\nlReZQVfupR+zWuwLU6bA+PFcMbp9nd/hRxqSEpG6oUDRQMRbzG/kSDis+Rd8wOG8yOlspBmnN3+V\nBQ+/BfvsA+gOX6Sh0TqKBiLaGoVyu8TNnw/XXQdPP82yRjtxQ/EI3srqw4hbmigIiKQhraNIU2WH\nj9q0CR7x7AsRs5jfTz/BwIHQuTO8/DJcfz1tfyrkQf8/vl2gICHS0DUJuwESv9Lho9JZSStWbP1b\n6VASRB4Cysqq3KNowiYG7/Ag5AwPpixdcAHcfDN06FA7/wARqZfUo6hHIk1dLSvWauXyU0ydU3iJ\nmdaV4auugL33hsmT4bHHFCREpBL1KOqReFYiR8pDwNZexj8GfEn/HwfwOz7gf+32hLH/DEq/JmGx\nnIikJ/Uo6pF41imYRclVLFhA3mt5TPrxAH7Xdhb8/e9sv+BrOPFEBQkRiUmBoh6JZ0tQ9wrDT//7\nHwwaFCSqX3gBBg+GwkK45BJoog6liFRNgaKeadmy6mMWLgQ2bYL772fDLjkwejSP/3IWB2XOJb/L\nSNhuu1pvp4ikD91S1hMVZzxF5/xx21dgr4EwZw6fNzqCqxjDV+wHi+DrGDOjREQiUY+inqhqxhNA\ndwp4j55MXHMKAH3aTuKI4neDIFFC+ziISKIUKOqJWDOedmEhj3M+BexPV2ZwGffDN98wftlJQOVE\ndbSZUSIikShQ1BORZjy1YjUjGcwcOnMmz3Irg8ihkFezL4WmTWncOPK5or0uIhKJAkU1xVOJNZ7j\n4z1P2RlPTdhEXx6gkBwGcyvP8Qd2Zy6DuZXNGdtvqeJaVBT5XNFeFxGJyN3r/aN79+5elyZOdM/I\ncA8mowaPjIzg9USO79s3wfM8Uey9207ymezhDv7jHof7ayO+9OxsdzP37Ozy783OLn/u0kd2dnK/\nDxGpn4ACj+MaG/pFPhmPug4UiV6Aox3fuHEC55k82b1nz+CA3Xd3f+kl9+LimO2sTkCLFnREJP3E\nGyg09FQNMSuxJvB6tCGgcsf/979Bsb7u3eGbb+Bvf4Pp0+GUU6pcUZ3IvhHx7lchIg2P9qOohrj2\ndojj+Giys2H+N2tg1Ci4886tUq0MAAAKhklEQVTgyn3llcFeEdtvX81Wx5bov0lE6j/tR1GLIpXS\niLUVaDylN0q1armZZ458EHJy4JZb4PTTYfbsIGjUUpCAxHtDItJwKFBUQ6JbgZY9PjrnT21f5b87\nduOAR/sGtZm++GLrtKhaFq3gYDyFCEUkvSlQVFNeXjAkU1wc/KyqJEbp8ZHSCnszlbfpxaPLfs/2\n22yGF1+E99+H/fevdGyi03LjlWgvSUQaDgWKBCTjIl32Dr093/Mof2IK+7Fvo2lw771BovrUUyNG\nlNpMOCfaSxKRBiSeqVGp/qiL6bGJTjWNdZ6dWq72EVzvP9PSN9DM72xyjf/joVXljok0TVXrIkQk\nmYhzemxKVo81s/nAGqAI2OxxZOVrW6SifKUF9uK+6968mby14zi96VBarl/KU5zD/e1vpe/ojpxV\nco6KVWLL7oWthLOIhCElp8eWBIpcd18ez/F1MT22UaPg/r0isyBPEZM7vP46XHMNzJwJhxwCY8bA\ngQdWOjTWNFXQFFYRSR5Nj02yas8KmjYNjjkm2HJ040Z4/nn48MOIQQJi9xqUcBaRMKRqoHDgTTOb\nbGYXh90YqMZFetEi6N0b9t0XpkyBu++GGTPIX386HXe1qAnx1q0jny4rSwlnEQlJPImMun4Avy75\nuRMwDTg8wjEXAwVAQVZWVhLTO9HFVQtpzRr3oUODTHezZu4DBrivXLnl/bES4hMnBm+pmKxu2lR1\nl0Qk+YgzmZ2SOYqyzGwYsNbd74h2TF2X8IioqAjGj4cbboAlS+Dss4OV1b/5zZZDqiqTEe3vmZmw\nPK5sjYhI/OptjsLMtjGzVqXPgWOA6bX9uTVaI/HGG7DPPsH0pN12g08/haefLhckoOpZS9H+vnJl\nAm0REUmylAsUwK+Aj8xsGvAF8Kq7v1GbHxhpIdv558Oll1bxxq+/hmOPheOPh/Xr4bnn4KOPoEeP\niIdXlRBXGQ0RSUUpFyjc/T/uvnfJo6u71/qcnkhrJNzhwQej9CwWL4Y+fYJexJdfwl13BdNezzgj\nZunvqhLimtUkIqko5QJFGKIN+bgHQWSLtWth2DDo1AmeeAKuugoKC4MS4M2aVfk5Vc1a0qwmEUlF\nKZ/MjkdNk9mx9oswg+JNRfDYY0Gi+ocf4Mwz4dZbYbfdyM8PgsnChcEQ0ciRurCLSP1Qb5PZYRg5\nMvqIUV7bN4O1EH/+c3CL//HH8MwzW4KEdoUTkXTXIANFxRlOAEceWf6YrkzndY7jiaXHBkNOzzwD\nn3wCBx+85ZhY9Z9ERNJFShYFrE3Riu61bBn8vjM/MIKh9GY8q9mOm3Ycww2zLoPmzSudS0X6RKQh\naHA9imi9gPUrfmYow5lHJy5kAvdyBbvxLTf+dHXEIAFVT2etrU2GRETqUloEim++if9iXPFuvxFF\nXMR45tGJ4QzjDY6jCzO5mrtYRWuysqJf8GNNZ1X+QkTSRjx1PlL9Ad3j3kyo7OY/vXjTp9LNHfzL\nJgf6kc0/qlSHqW/fquszaZMhEamPSJdaT/Ewy/WgPmAg1v4M+flw95+nM3zDQE7gdb6jIzc2G8Wx\n484Cs0pTXYcMqd4eEDXav0JEpA7EOz02LQNF1Ivxjz/C0KEUPzKOtbRihN/ApKy/cuMtzaOufaju\nBb+qAoAiImFr0OsoKiWZ162Dm26CnBx49FEaXXE52y37lju8P3MXRA8SEc9VxeulVI5DRNJF2gWK\nchfjopIV1Z06wdChQQG/mTODTYQyM+M6X3Uv+CrHISLpIi2Gnpo3z/VNmwrKl9B45x0YMACmToX9\n9w/2qD7ssGqdX2U6RCQdNagcRblaTzNmwMCB8NprwW38qFFw1llBskFERLZoeDmKJUvgkkugW7eg\nHtNtt8Hs2XDOOQoSIiI1kB5X0B9+CBLV48bBZZfx3KhCOt5/DY0yWmhFtIhIDaVHoFi8GHr1ghkz\nyD/wXi7s36bSiuhLL1U5DRGR6kiPHEXnzl4wZw4Qff2CWfn1EBkZmoUkIg1bw8pRtGq15Wms3erK\nUjlwEZH4pEegKKOqhXBlqRy4iEjV0i5QRFogF233ukSCiohIQ5V2gSLSiuhLLlE5DRGR6kq7QAFB\nsJg/PyjaN38+PPCAymmIiFRXg9kKNS9PgUFEpDrSskchIiLJo0AhIiIxKVCIiEhMChQiIhKTAoWI\niMSkQCEiIjEpUIiISEwKFCIiElNalBk3s2VAhOLidaoNsDzkNqQKfRdb6bvYSt/FVqnyXWS7e9uq\nDkqLQJEKzKwgnrruDYG+i630XWyl72Kr+vZdaOhJRERiUqAQEZGYFCiSZ2zYDUgh+i620nexlb6L\nrerVd6EchYiIxKQehYiIxKRAUQvMbICZuZm1CbstYTGz281stpl9bWYvmtkOYbeprpnZcWY2x8wK\nzWxQ2O0Ji5ntYmbvmdksM5thZv3CblPYzKyxmX1lZv8Muy3xUKBIMjPbBTgaWBh2W0L2FvBbd+8G\nzAWuC7k9dcrMGgP3A8cDXYBzzaxLuK0KzWagv7vvCfQALmvA30WpfsCssBsRLwWK5LsLGAg06OSP\nu7/p7ptLfv0M6BBme0JwAFDo7v9x943A08ApIbcpFO7+g7tPKXm+huAC2T7cVoXHzDoAJwKPhN2W\neClQJJGZnQwscvdpYbclxfQGXg+7EXWsPfDfMr9/TwO+OJYys47AvsDn4bYkVHcT3EwWh92QeDWY\nPbOTxczeBnaO8KchwGDgmLptUXhifRfu/nLJMUMIhh7y67JtKcAivNage5lmti3wPHClu68Ouz1h\nMLPfA0vdfbKZHRF2e+KlQJEgd+8V6XUz2wvYFZhmZhAMtUwxswPc/cc6bGKdifZdlDKzC4HfA0d5\nw5uH/T2wS5nfOwCLQ2pL6MysKUGQyHf3F8JuT4gOAU42sxOAFsB2ZjbR3c8LuV0xaR1FLTGz+UCu\nu6dC4a86Z2bHAXcCv3P3ZWG3p66ZWROCJP5RwCLgS+CP7j4j1IaFwII7pwnASne/Muz2pIqSHsUA\nd/992G2pinIUUlv+BrQC3jKzqWb2YNgNqkslify/Av8iSN4+0xCDRIlDgPOBI0v+W5hackct9YR6\nFCIiEpN6FCIiEpMChYiIxKRAISIiMSlQiIhITAoUIiISkwKFiIjEpEAhIiIxKVCIiEhMChQiIhKT\nAoVIEplZEzP72MzWmtkeFf52ccnOhyPCap9IdaiEh0iSmVk2MBVYAPRw9w1m1pWgMOBk4Ah3Lwqz\njSKJUI9CJMncfQHQB9gbuMPMWhLscLcByFOQkPpGPQqRWmJmDwB9gU+Ag4EzGvheDFJPKVCI1BIz\nawFMB3YDHnb3i0Nukki1aOhJpPZ0A7JKnv+2ZDMjkXpHgUKkFpjZdgR5ieUE+6kfBAwPtVEi1aQ7\nHJHa8RCQDRzt7u+a2T7AIDN7293fC7ltIglRjkIkycysD/AIcIu7Dyl5bQeCKbNNgW7uviLEJook\nRIFCJIlKFtlNJggKvyvZO7v0bwcBHwCvu/vJITVRJGEKFCIiEpOS2SIiEpMChYiIxKRAISIiMSlQ\niIhITAoUIiISkwKFiIjEpEAhIiIxKVCIiEhMChQiIhKTAoWIiMT0/xDN1ForFzSDAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x330c9b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#TODO 请选择最适合的直线 y = mx + b\n",
    "# m = 0\n",
    "# b = 0\n",
    "\n",
    "m = 2.46\n",
    "b = 14.4\n",
    "\n",
    "# 不要修改这里！\n",
    "plt.xlim((-5,5))\n",
    "x_vals = plt.axes().get_xlim()\n",
    "y_vals = [m*x+b for x in x_vals]\n",
    "plt.plot(x_vals, y_vals, '-', color='r')\n",
    "\n",
    "plt.xlabel('x',fontsize=18)\n",
    "plt.ylabel('y',fontsize=18)\n",
    "plt.scatter(X,Y,c='b')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.2 计算平均平方误差 (MSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "我们要编程计算所选直线的平均平方误差(MSE), 即数据集中每个点到直线的Y方向距离的平方的平均数，表达式如下：\n",
    "$$\n",
    "MSE = \\frac{1}{n}\\sum_{i=1}^{n}{(y_i - mx_i - b)^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.891403183774\n"
     ]
    }
   ],
   "source": [
    "# TODO 实现以下函数并输出所选直线的MSE\n",
    "\n",
    "def calculateMSE(X,Y,m,b):\n",
    "    sum_squared = 0\n",
    "    mse = 0\n",
    "    n = len(X)\n",
    "    for x, y in zip(X, Y):\n",
    "        sum_squared = sum_squared + (y - m*x - b)**2\n",
    "    mse = sum_squared/n  \n",
    "    return mse\n",
    "\n",
    "print(calculateMSE(X,Y,m,b)) \n",
    "# #---------------------------------------\n",
    "#print(calculateMSE(X,Y,2.5, 14))\n",
    "#print(calculateMSE(X,Y,2.5, 14.2))\n",
    "#print(calculateMSE(X,Y,2.5, 14.4)) # b = 14.4时最小\n",
    "#print(calculateMSE(X,Y,2.5, 14.6))\n",
    "##-------------------------------\n",
    "# print(calculateMSE(X,Y,2.5, 14.4))\n",
    "# print(calculateMSE(X,Y,2.48, 14.4))\n",
    "# print(calculateMSE(X,Y,2.46, 14.4)) #m = 2.46时最小\n",
    "# print(calculateMSE(X,Y,2.44, 14.4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.3 调整参数 $m, b$ 来获得最小的平方平均误差\n",
    "\n",
    "你可以调整3.2.1中的参数 $m,b$ 让蓝点均匀覆盖在红线周围，然后微调 $m, b$ 让MSE最小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 (选做) 找到参数 $m, b$ 使得平方平均误差最小\n",
    "\n",
    "**这一部分需要简单的微积分知识(  $ (x^2)' = 2x $ )。因为这是一个线性代数项目，所以设为选做。**\n",
    "\n",
    "刚刚我们手动调节参数，尝试找到最小的平方平均误差。下面我们要精确得求解 $m, b$ 使得平方平均误差最小。\n",
    "\n",
    "定义目标函数 $E$ 为\n",
    "$$\n",
    "E = \\frac{1}{2}\\sum_{i=1}^{n}{(y_i - mx_i - b)^2}\n",
    "$$\n",
    "\n",
    "因为 $E = \\frac{n}{2}MSE$, 所以 $E$ 取到最小值时，$MSE$ 也取到最小值。要找到 $E$ 的最小值，即要找到 $m, b$ 使得 $E$ 相对于 $m$, $E$ 相对于 $b$ 的偏导数等于0. \n",
    "\n",
    "因此我们要解下面的方程组。\n",
    "\n",
    "$$\n",
    "\\begin{cases}\n",
    "\\displaystyle\n",
    "\\frac{\\partial E}{\\partial m} =0 \\\\\n",
    "\\\\\n",
    "\\displaystyle\n",
    "\\frac{\\partial E}{\\partial b} =0 \\\\\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "### 3.3.1 计算目标函数相对于参数的导数\n",
    "首先我们计算两个式子左边的值\n",
    "\n",
    "证明/计算：\n",
    "$$\n",
    "\\frac{\\partial E}{\\partial m} = \\sum_{i=1}^{n}{-x_i(y_i - mx_i - b)}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{\\partial E}{\\partial b} = \\sum_{i=1}^{n}{-(y_i - mx_i - b)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO 证明:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.2 实例推演\n",
    "\n",
    "现在我们有了一个二元二次方程组\n",
    "\n",
    "$$\n",
    "\\begin{cases}\n",
    "\\displaystyle\n",
    "\\sum_{i=1}^{n}{-x_i(y_i - mx_i - b)} =0 \\\\\n",
    "\\\\\n",
    "\\displaystyle\n",
    "\\sum_{i=1}^{n}{-(y_i - mx_i - b)} =0 \\\\\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "为了加强理解，我们用一个实际例子演练。\n",
    "\n",
    "我们要用三个点 $(1,1), (2,2), (3,2)$ 来拟合一条直线 y = m*x + b, 请写出\n",
    "\n",
    "- 目标函数 $E$, <br />\n",
    "$E = \\frac{1}{2}[(1 - 1m - b)^2 + (2 - 2m - b)^2 + (2 - 3m - b)^2]$\n",
    "- 二元二次方程组，\n",
    "- 并求解最优参数 $m, b$\n",
    "    $$\n",
    "    \\begin{cases}\n",
    "    \\displaystyle\n",
    "    -1(1 - 1m - b)-2(2 - 2m - b)-3(2 - 3m - b) = 0 \\\\\n",
    "    \\\\\n",
    "    \\displaystyle\n",
    "    -(1 - 1m - b) - (2 - 2m - b) - (2 - 3m - b) = 0 \\\\\n",
    "    \\end{cases}\n",
    "    $$\n",
    "    \n",
    "    $$\n",
    "     --> \\begin{cases}\n",
    "    \\displaystyle\n",
    "    m = \\frac{1}{2} \\\\\n",
    "    \\\\\n",
    "    \\displaystyle\n",
    "    b = \\frac{2}{3} \\\\\n",
    "    \\end{cases}\n",
    "    $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO 写出目标函数，方程组和最优参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.3 将方程组写成矩阵形式\n",
    "\n",
    "我们的二元二次方程组可以用更简洁的矩阵形式表达，将方程组写成矩阵形式更有利于我们使用 Gaussian Jordan 消元法求解。\n",
    "\n",
    "请证明 \n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    \\frac{\\partial E}{\\partial m} \\\\\n",
    "    \\frac{\\partial E}{\\partial b} \n",
    "\\end{bmatrix} = X^TXh - X^TY\n",
    "$$\n",
    "\n",
    "其中向量 $Y$, 矩阵 $X$ 和 向量 $h$ 分别为 :\n",
    "$$\n",
    "Y =  \\begin{bmatrix}\n",
    "    y_1 \\\\\n",
    "    y_2 \\\\\n",
    "    ... \\\\\n",
    "    y_n\n",
    "\\end{bmatrix}\n",
    ",\n",
    "X =  \\begin{bmatrix}\n",
    "    x_1 & 1 \\\\\n",
    "    x_2 & 1\\\\\n",
    "    ... & ...\\\\\n",
    "    x_n & 1 \\\\\n",
    "\\end{bmatrix},\n",
    "h =  \\begin{bmatrix}\n",
    "    m \\\\\n",
    "    b \\\\\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO 证明:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "至此我们知道，通过求解方程 $X^TXh = X^TY$ 来找到最优参数。这个方程十分重要，他有一个名字叫做 **Normal Equation**，也有直观的几何意义。你可以在 [子空间投影](http://open.163.com/movie/2010/11/J/U/M6V0BQC4M_M6V2AJLJU.html) 和 [投影矩阵与最小二乘](http://open.163.com/movie/2010/11/P/U/M6V0BQC4M_M6V2AOJPU.html) 看到更多关于这个方程的内容。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 求解 $X^TXh = X^TY$ \n",
    "\n",
    "在3.3 中，我们知道线性回归问题等价于求解 $X^TXh = X^TY$ (如果你选择不做3.3，就勇敢的相信吧，哈哈)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2.4531, 14.3983)\n"
     ]
    }
   ],
   "source": [
    "# TODO 实现线性回归\n",
    "'''\n",
    "参数：X, Y\n",
    "返回：m，b\n",
    "'''\n",
    "\n",
    "def linearRegression(X,Y):\n",
    "    #为了不改变X和Y, 把X和Y的数据结构转换给A和B,然后对A和B操作\n",
    "   \n",
    "    A = [[x,1] for x in X]\n",
    "    B = [[y] for y in Y]\n",
    "        \n",
    "    #求A的转置AT\n",
    "    AT = transpose(A)\n",
    "    \n",
    "    #求矩阵相乘ATA,A的转置AT乘以A\n",
    "    ATA = matxMultiply(AT, A)\n",
    "    \n",
    "    #求矩阵相乘ATB\n",
    "    ATB = matxMultiply(AT, B) \n",
    "\n",
    "    # 用高斯消元法求解向量 h \n",
    "    h = gj_Solve(ATA, ATB)\n",
    "                                       \n",
    "    return round(h[0][0],4), round(h[1][0],4)\n",
    "    \n",
    "\n",
    "m,b = linearRegression(X,Y)\n",
    "print(m,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你求得的回归结果是什么？\n",
    "请使用运行以下代码将它画出来。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAESCAYAAADjS5I+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd0VNXax/HvphOxEUBqJqKhS1cR\nUfQiFlTsWAJy1fvalaKogF6xRBGU4rWLBYErFkDsCorYUAgIKCBEgSBgoXilIyTP+8dJQsrMZCaZ\nZCbJ77PWrJAzZ87syWKdZ/Z+9n62MzNEREQCqRTtBoiISGxToBARkaAUKEREJCgFChERCUqBQkRE\nglKgEBGRoBQoREQkKAUKEREJSoFCRESCqhLtBkRCnTp1LDExMdrNEBEpUxYuXLjZzOoWdl65CBSJ\niYmkpqZGuxkiImWKcy49lPM09CQiIkEpUIiISFAKFCIiEpQChYiIBKVAISIiQSlQiIiUY1OmQGIi\nVKrk/ZwyJfxrlIvpsSIiUtCUKXDttbBrl/d7err3O0BycujXUY9CRKScGj78QJDItmuXdzwcChQi\nIuXUunX+j6ene0NR0PaYUK6jQCEiUk4lJAR+zgygarVQrqNAISJSTqWkQFxc8a+jQCEiUk4lJ8Nz\nz4HPB84dOH4EvzGBa0K+jgKFiEg5lpwMa9dCZiY0S9jDXTxMGkn0Y1LI11CgEBEpg8JaH2EGb7zB\nwt0teZhhzOY0WrE85PdSoBARKWOy10ekp3sxIHt9hN9gsXAhnHwy9OlDrQaHMHvoJwzyzWC1OxrY\n93co7+fMS32XaZ07dzbtRyEiFUViohcc8vP5vGEmAH791Vsw8fLLUKcOPPggXHMNVK6cc75zbqGZ\ndS7s/bQyW0SkjAm0PmLdOmD3bhgzBh5+GPbtgyFDYNgwOPTQIr+fhp5ERGJMYfkH/+sjjBvjX4MW\nLeDuu+GMM2D5cnjkkWIFCVCgEBGJKaHkH/Kvj+jMAr6qdBJPbL4MDj8c5syBadPgqKMi0iYFChGR\nUlRYbyGU+kzZ6yOObbSRl+nPAo6jQ600mDDBS16fckpE26wchYhIKQmlmmvQ/EO23btJXvMYyX8+\nDNX2w6A7qTlsGBxySIm0O2o9CudcE+fcHOfcCufcMufcgKzjI5xzG5xzi7MevaLVRhGRSAqltxCo\nPlNCAt5Y1NSp0Lw53HMPnHUWrFgBI0eWWJCA6A497QduM7OWQBfgJudcq6znxppZ+6zH+9FroohI\nYOFuChRKb8Fffaa4OHjm6vlw4olw+eXedNfPPoM334SmTYvxCUITtUBhZr+a2aKsf28HVgCNotUe\nEZFwhLXoLUvQ3kKW/PWZjm+0nu87XsmZ9x4Pq1fDCy/AggXQvXuethR3F7ugzCzqDyARWAccAowA\n1gJLgReBwwt7fadOnUxEpDT5fGZeiMj7iI/3nnPO+zl58oHXTJ5sFheX9/y4uLzn5Ni50+y++7wT\nqlc3GzrUbNu2AqeFdc18gFQL5R4dykkl+QBqAQuBC7N+PwKojNfbSQFeDPC6a4FUIDUhIaHwv4iI\nSAQ55z9Q5H/kv2lPnhw4kJiZWWam2ZQpZo0bexe45BKz1asDtiNQwPL5Cv8MoQaKqJbwcM5VBd4F\nPjKzMX6eTwTeNbM2wa6jEh4iUtoCldHwJz4eatXychEJCV4ewu+e1d98A4MGeT87doRx4+Ckk4Je\nu1Kl7E2I8nLOqxgbTKglPKI568kBLwArcgcJ51yDXKddAPxQ2m0TESlMOJsCbdkSPJcx4/FfmFGr\nL5xwAn8sWMu8a1/y8hCFBAkILe9RXNGc9XQi0A/4R76psKOcc98755YCpwKDothGERG/8iedfT6v\n5xCKnCmxO3ey9MIRnDGgOWftfJMHGU7TjDROm/xPprwa2u050CyplJTwPk8wqh4rIhIh+RfUBeLI\nJJn/MqnRXbBhA1O5lDt5hHX4cs7JUwk2hPcdPjyEoa387VD1WBGR0pN9s961y6vknZHh3ex37PCG\nnrJ1YR7jGMjxzIcGnem24TW+4sQC1wu05sKf5OTQAkNRqdaTiEgY/K1ZyL2mArwgkT38M3689+8m\nrGMKVzCPriS4X/j6uonw7bf8GF8wSEBkcwzFpUAhIhVaOIvVAi2yGzAgcGmO5PN28M3p/2aVa84F\nzGD8offw+fOr6PrMlUx5tRLbthV8n2rVIptjKC7lKESkwvKXU4iL85LU/oZywpkS68ikL5MZVXko\n9TM2MjPucjIfGskFAw50FerUyTsslS0+HjZvDu+zFEXMT48VEYm2UIr05RZq3qArX/Etx/MK/UnP\naExXvuL8Xf+l77CEnB7LlCn+gwTA1q2hvU9pUaAQkQqrsCJ9+Yelatf2f358vNcTSSCdV7mMr+hG\nA36lL5M4gXnMoyuQNwgFCkYQW/kJUKAQkQos2GI1f/mIbdu8/EFucXHw5CM7+Lbn3ax0LejN24w7\n9F6as5Ip9MXy3Wazg1Cw3kks5SdAgUJEKrBAq6t37PCfoN63Dw4++MAiu8SETGb3fZlL72lGm5kp\n1LjiIuLWrWTg/0ZQ13eQ3/fMDk6BglR8fMlOdS0KBQoRqbCyV1fnX1G9ZUvw/MHatZA59wvW1D2O\nE567yrvrz5sHkydDkyZA4SumAz0/fnzxP1ekKVCISIWWnOwV7AtV14ZroU8fOPlkNi7+nWSmUDft\na+qc0yXPFFt/JT5yz6Yq7PlYoumxIlLhBarAmlsttnNPlZHc5h7DXCVG2p08tG8Iuyk4dhVsim0s\n0fRYEZEQBcsXJCZkchUv8XPlZtyx/yEqX3oJ3equ4p599/oNEhB8im1ZpEAhIhVeoHzBqzd8zpr4\nzrzI1dQ7NtHbJ2LSJOZvbFzoNcOp1RTrFChEpMLLny/o1nA1K1pfTM8Hu3tLpF99Fb7+Go4/Hght\nnUOsrYUoDgUKERG8YLF26TYy7xzKF5tbkrDsA3jgAfjxR7jsMi+CZCls06JI7wcRbQoUIiIZGfDC\nC9CsGYwc6QWGVavg7rv9RoT8PZD4eO8R67OXikr7UYhIxfbZZ94+1YsXQ9eu8M47cOyxhb6spPeA\niCXqUYhIxfTzz3DRRXDqqd4quqlT4csvQwoS4ZQmLw/UoxCRimXbNi+BMG4cVK0KDz4IgwdDzZoh\nvTx/afLsPSmg/PYw1KMQkYohIwOefx6SkmDUKLjiCi8PMXw4U6bXDLmHEG5p8vJAgUJEyp38Q0Oz\nh8+Bjh29r/7NmkFqKrz0EjRsGHDXukDBorDS5OWRAoWIlCu5b/xN7SfGpl/AaQ/9gx0b/4LXX4fP\nP4dOnXLOD7eHEKw0eXmlQCEi5crw4VBl11+MYgjLacVpzGYoD9Ep7ke45JI86yEg/B5CYVVhyyMF\nChEpP/bv58z0Z0kjidt4jEn0oxmrGMlQ0n6p4fcl4fYQylLV10hRoBCR8uGTT6BjR57helbQks6k\n8i9e4DcaAIFv/EXpISQnZ+1Jken9LM9BAhQoRKSElNpag7Q0OO88OO002L6dz299k141P+M7OuY5\nbccO/22oiD2EsJlZmX906tTJRCR2TJ5sFhdn5s0j8h5xcd7xiPnzT7PBg82qVjWrVcts5Eiz3btz\n3j8+Pu/7l0gbyjgg1UK4x6pHISIRVxJrDbJ7KFXdfu6Of5o9CUkwdiz07+/1Ku68E2p4eYhAu9aV\n9/UOJUUrs0Uk4iK91iB7ymvXXbN4l0G02bqMzyt1Z8eD4+g1rH2ptKEiU49CRCIu0msNXrhjJVN3\nncssTqcmu7mA6XTPnMONz/kPEiXRhoosaoHCOdfEOTfHObfCObfMOTcg63ht59ws51xa1s/Do9VG\nESmaiK01+PNPGDSIjza2oTtzGcIoWrGct7gAcKSnF78NFa3AX5GEksgoiQfQAOiY9e+DgVVAK2AU\ncFfW8buARwq7lpLZIrFn8mQzn8/MOe9noCSy3/P27TN74gkvI+2cPeuutXr8ViA5Xbly8dpQKkn3\nGEaIyeyoz1jKaQjMBHoCK4EGdiCYrCzstQoUIrElnCCR/0bdu/qH9mejVt4vp55qtnhxgQCR+1Ec\nPp//a/p8xbtuWRFqoIiJHIVzLhHoAHwLHGFmvwJk/awXvZaJSLjCKbKXe3ZUc37kXc5m5t4z+d/v\ne+Gtt7xFdO3a4fP5fy9/x8MZSlLCO0ShRJOSfAC1gIXAhVm//y/f838GeN21QCqQmpCQENkwKyJF\nFs63dOfMDmeLjeNW+5sq9j8OscE8atXZk+e8UIeIwh1KUo+iDAw9AVWBj4DBuY5p6EmkDHPO/83X\nuXwn/v23/fvwx20Lh9t+KtnTXGd1+T3gjTqU4axwb/zKUcT40JNzzgEvACvMbEyup94G+mf9uz9e\n7kJEyohA008rVTowHPTpHR9Cu3bc9+etLKnUkQ58xw08wybqBZwdFUp9pXCHklS+IzTRzFGcCPQD\n/uGcW5z16AWMBHo659Lwktsjo9hGEQmTv2mp4G0w19xW8FR6L/4x+iy2bd0HM2eyceIstvnaRuRG\nXZS1ExWtwF9RRG1ltpl9CbgAT/cozbaISORk32gHDIAtW7x/12YL93IfN/IUO6jFYB7jneo3k9a7\nGslAct/IvHdKSt79rKH87xVRGmJi1pOIlD+7d0MV9nEr40kjiZt4kme5jiTSGMtgfv6lWsTfU0NJ\nJcN5+YyyrXPnzpaamhrtZohIlkSf0WrdB4xhMC1YySxOYxBjWUabnHN8Pm+oR6LHObfQzDoXdp56\nFCISWcuW8cy6s3ifs3EY5/AOp/NxniCh4aCyRYFCRCJj82a4+WZo144TKn3LQMZyDN/zHueQOx2p\n4aCyR2XGRaR4/v4bnnoK7rsPtm+H669nVpsRPH9bHfblSyorQJRN6lGIiF+FlsIwg3ffhWOOgUGD\n4LjjYMkSeOIJLr6+jpLK5Yh6FCJSQHa9puxpptn1miDrZv/DDzB4MMyaBc2bw3vvwVlneVGBA+cp\nMJQP6lGISAGBtjJ97K5NcOON0K4dpKbC+PHw/ffQq1eeICHliwKFiBSQv+RFVf5mEGP4dH2SN4Z0\n003ePtW33gpVq0ankVJqFChEYkQs7bR2oOSFcS5vs4zWjOE2vqvR1etBPP44xMdHr4FSqhQoRGJA\nOHs4lIaUFDiuxlJm0ZO3OY99VOWC6u+zccL70LJlgbbHSoCTkqFAIRIDAuUEhg+PQmP++IPkL65n\n3t4OdKr0HbfwH3onLOHiF84qkJyOtQAnJUMlPERiQKVK3o02P+e8qqalYu9e+M9/4IEHvCh1003w\n739D7doBX5KY6AWH/FSeo2xQCQ+RMqQo5bGLwu8wkRlzB73F2lqtYcgQPtl3Eu889D2MGxc0SIC2\nEq0oFChEYoC/PRwiXQ/J3zDRf/61hJWNe9B93AXs3F+dM/iQ03a/y2UjWoQ0fFRaAU6iS4FCJAaU\nRnns3HmQevzOs1zL13s6EL9xKTfyJO1YwsecAXjnDRhQ+DVLI8BJ9ClQiMSIkt5pbd06qMZehjCK\nNJK4ipcYx0CSSONpbiQjX6GGLVsKT0pr/4eKQclskYrAjOuOeIs7Nt3OUazmHc7hNh4jjWZBXxYf\n7xWFlfJJyWwR8Xz3HZx6Ks9uupC9riY9+ZjevFNokIDQehVS/ilQiJRXv/0G//oXdOoEy5bB00+z\n+OXFzKncM6zLRGUth8QUVY8VKW/27PGmtqakeGsjBg+Gu++Gww7jCsAq560MC1CtmrethD+a6irq\nUYiUUQXWREw2mDYNWrWCoUOhRw+vJ/Hoo3DYYTmv85eAfvHFwKWbNNVVFChEyqD8ayJqpy8iof8p\ncPHFcNBBMHs2Uy55i8SeSX5rMPmbYTV+vKa6in8KFCIxKlixvew1EfX5lQlcQyqdaZG5nGG1n4Hv\nvmPKbz3CrsGkqa4SiKbHisSg/DvMQd49p2u6PQxkLMN4iOrsZTwDeJC72e4OJTNTNZgkNKFOj1Uy\nWyQGBawmO8xIrvYmaZWH0DgjnRmczxBG8zNHA+DLyieoBpNEkoaeRGKQvxt6RxYyad3J0KcPW+0w\nTuVTLmRGTpDInU9QDSaJJAUKkRiU+4begI28yFUs4Fias5L/4zk6ZC7kM07NOSc+Pm8+QTWYJJIU\nKERiQP7Eda9eULvmboaRwiqacQX/ZTRDSCKNCfwfmVTO8/patfImnZWYlkhSjkIkyvInrtPTjR0v\nvM6qancQzzqmcSF3MIrVHBXwGv6GqpKTFRgkMqLWo3DOveic+8M590OuYyOccxucc4uzHr2i1T6R\nUERiv+jcievOLOBLuvHK35fx297aMGcOt/mmBQ0SoNyDlKxoDj29DJzp5/hYM2uf9Xi/lNskErJI\n7Re9bh00ZAMv058FHMdR/Mw1TKDdvlQ45RS/+YbclHuQkha1QGFmnwNbo/X+IsUVcAprOEX0du1i\n9CEPsIpmXMZUHuYumrGKF7mGxj4vD5E/3xAf7z2Ue5DSEos5ipudc1cCqcBtZvZntBsk4k+x1iqY\nwdSpcOed3PbXL0yvfDG3ZzzCGpoCBXsJyjdINMXarKengaOA9sCvwGOBTnTOXeucS3XOpW7atKm0\n2ieSo8hrFb79Fk48Ea64AurUgblz2T3xDTJ9TdVLkJgUVqBwzq1yzt3pnKtfEo0xs9/NLMPMMoHn\ngeOCnPucmXU2s85169YtieaIBBX2WoX166FfP+jSBdas8Uq2LlgAJ5+cp0hfSoo3fFWcBLlIJIXb\no9gHPAysc8695Zw7xzkXsV6Jc65Brl8vAH4IdK5ItIW8VmHXLrjvPmjWDN54A4YNg1Wr4KqroHLe\n9RCRSpCLRFJYN3kzaw10BSYCpwIzgV+ccynOueDz9/Jxzr0KzAOaO+fWO+euAUY55753zi3Nuv6g\ncK4pUpKmTPFGipzzHnXqeMfzl+vOkZnpvah5cxgxAs49F3780esyHHyw3/eISIJcJMKKXD3WORcH\nXApcgxc8DJgLTACmmdneSDWyMKoeKyVtyhS4+uqCu8BVqgSvvOKnF/HNNzBwoJeP6NQJxo6Fk04q\n9H0qVfJ6Evk558UdkUgKtXpskYeNzGyXmb1kZt2AFsBU4BRgErDROTfWOadlQFIuDB/uf6vQzEwY\nMCDXgV9+8aLGCSd4059efhnmzw8pSICK+UlsKlZ+wTlX2Tl3ATAGr3dhwBzgG+AWYIVz7rxit1Ik\nyoJNed2yBdi5E+691xtmmj7d26N61Sro39/rJoRIxfwkFhUpUDjnWjjnRgMbgGlAZ+BRoJmZnWZm\nZ+P1MlYCoyLVWJFoCfSN3pFJXyZ5ier774fzzvPyEA884FXqC5OK+UksCmvBnXPuarycRJesQ7OB\n54CZZrY/97lm9pNz7nG8nIVImZaSAn375j3WhXmMYyDHMx8adobXX/fWRxSTFtdJrAm3RzEBOBIY\nCRxlZmeY2bT8QSKX5Xg5C5EyLTkZbrjB+3cT1jGFK5hHVxqznq+vm3hgEV2YIlFUUKSkhVvC4yLg\nbTPLCOVkM5sPzA+7VSIx6KlRO7hh4yMkvf0oZjD+0Huo/9gdXHpN+ENM4K+8uPc7qEchsaXI02Nj\niabHSonKzIRJk2DoUPj1V6/0xsMPF3sqUmKiFxzy8/m8NRkiJa3Ep8eKVAhffgnHHw///Cc0aQJf\nf+11BbKCRHGGjopVVFCkFClQiPiTng6XXuqtf/j1V69HMW+etz4iS3HLbWjNhJQVChQSk6KW5N2x\nw1sD0bw5vPOOtzZi5UpvylO+9RDFLbehNRNSVihQSMwJ95t6RIJKZqa3ijopCVJSWNPpYk6ovZJK\n948gsfVBfq9Z3KEjrZmQMsPMyvyjU6dOJuWHz2fmhYi8D5+v4LmTJ5vFxeU9Ly7OOx6yzz8369jR\ne/Hxx9sHI+aFdM1w2ikSi4BUC+Eeqx6FxJxwvqkXa/hnzRro0wdOPhn++MPrisybx/UvdQnpmho6\nkopCgUJiTjhJ3kBBJT09yDDU9u3enhAtW8J773l7Raxc6U17dS7kQKWhI6koFCgk5oTzTT3YDKEC\nuY2MDG9XuaQkbx1Enz5egPj3v/O8YTiBKvfOdAX2oxApJxQoJOaE803dX1DJLWfIaO5cOPZYuOYa\naNrUK7nxyivQuHFI19SQklRkChQSkwr7pp4906lfP6hZE+Lj/V/nSFbzaPrFcMopsHkzvPoqfPUV\nHBdwO3YNKYnkE26tJ5Goy18jacsW7xt/fHzW3hDAwWxjGA8xiLFkuCpw/wNw221eVAmBKriKHKBA\nIWVOoJlONWtCrZoZXLr7JVIYzhH8weTK/Ykb9xAX3twwOo0VKQc09CRlTqBZScds+Yz0up2YwP+R\nRhK968/HTXxZQUKkmNSjkDInISFv1dWm/MxohnAhM8D54LXX6HbJJXRzLnqNFClH1KOQqCpK+Y3s\nWUmH8BePcAfLacXpfMziS1JgxQpv2mtWkNDGQCLFpx6FRE1RN+5JviyDoz99gaNevpvamZt586B/\nUnlkChfd3CAi1xeRvLRxkURNkTbu+fRTGDQIli6Fbt1g3Djo1Cly1xepQLRxkcS8sKqvpqXB+edD\njx6wbRu88QZ8/nnAIBH29UUkIAUKiZqQSmX8739w++3QujV88olXemPFCrj44pw8RLGuLyKFUqCQ\nqJgyxVso7U+vXsD+/fDMM15dpjFj+KnrlRx7WBqVht1FYosaYSW9c1MpDpHwKVBIqctOMu/c6f/5\nv6bNhg4d4IYboFUr3n9gIe0WTCB1ff2wthxVKQ6RyFAyW0pdoCTz0aTxGLfRm3fgyCNh9Gi48EIS\nj3RKSouUACWzJWblTyYfyv94lNtYRmtO4TNGHjYSli+Hiy4C5z9IgP9gIyKRF7VA4Zx70Tn3h3Pu\nh1zHajvnZjnn0rJ+Hh6t9knJyU4mV2Y/1/M0P3E0gxjLRPrTvuYqmjxxJ9SokXN+5cr+r5O9iE6L\n6URKVjR7FC8DZ+Y7dhfwiZklAZ9k/S4xprirnVNS4NzqH7OY9jzNjXzPMXRkEUPjn+eB5+sXyCFk\nZPi/Tmam16sIJ28hIuGLWqAws8+BrfkOnwdMzPr3ROD8Um2UFCo7EV3kG/TKlSS/eg5v7z2DWlX2\ncAEzuCrhU4ZMbs/mzf4TzT5faJcOea9sEQlLrOUojjCzXwGyftaLcnskn0Alvgu9Qf/5p7eiuk0b\n+OILGD2axB3LmGHnszbdBZ2JVNgudrlpMZ1I5MVaoAiZc+5a51yqcy5106ZN0W5OhRH2auf9++HJ\nJ+Hoo+Hxx+Hqq71V1rffDtWrFzjd37CWv2mugXa002I6kciLtUDxu3OuAUDWzz8CnWhmz5lZZzPr\nXLdu3VJrYEUX1mrnjz6Cdu3g5pu9n4sWwbPPQj3/HcVgw1r5t0YdP16L6URKS6wFireB/ln/7g/M\njGJbxA9/w0DOZa2mzvbjj3D22XDmmbB3L7z1lld+o127oNcOZ1hLi+lESk80p8e+CswDmjvn1jvn\nrgFGAj2dc2lAz6zfJYYkJ0P//nnLLJnBxInwxrNbYcAALw/x5Zfw6KOwbBmcd16hdZkg+LBWoCGp\n3L0MBQmRkqGV2RK2/Curq7CP63mGByrdy2H85Y0X3X8/hDkkGGjFdnw87N6dt7cRF6cehEhxaWW2\nlJjc3/zP5AOW0pb/cCupmR1h8WJ4+umwgwQELuIHRZxpJSIRoUAhBRS2oK52bWjJct7nLD6gF5XJ\n4Fze5rLas+CYY4r8voHyDlvzr7bJoqmwIqVDgULyKHRB3ZYtPLzzFpbSlhOYxyDG0IYfeJdzQ8pD\nFMZf3kH7SohElwKF5BFo5tG9w/Z5c1KPPpqr9zzFs1zH0fzEOAaxj2pA4G/+xaV9JUSiS4FC8ig4\nnGP04j3eXXcMDBwIxx7LWQ2WcDNPsoU6ec4sqW/4mgorEl0KFBVIKMX8ct/sW7GMDzmT9ziHqlUM\n3n0XPvqI/qPblPo3fE2FFYkeBYoKItRifikp0KTmZp7gJpbQjuOYz5Cq45g/4XtvEZ1z+oYvUsFo\nHUUFEWiNQp5d4v7+G558kr/vvo9Ku3bwDDcwofEIhoyMVxAQKYe0jqKcKupeEEGL+ZnBO+94K6oH\nD6baSV2osmwpN9t/WPyLgoRIRadAUYb4Gz7q188b/iksaARKNPes/z2cfjr07u1tJff++/Dhh9Cq\nVYl8BhEpexQoyhB/U1ezRw4L20Ao/xTTOmzi2So38sFv7WHhQq8E+NKlcNZZJdN4ESmzFCjKkMJW\nIgcra5GdgD464W9u4zF+dkn8K/M5Kt1yM/z0E9xyC1StGvlGi0iZVyXaDZDQJST4T0jnFvB5M5Jr\nvU1ytduBn+CsXl5115YtI91MESln1KMoQ0LZEtQ5P8NPS5fCaafB+ed7vYYPPoD33lOQEJGQKFCU\nMTVrBn/eLNfw0x9/wHXXQYcOXlXXJ56AJUu8DYVEREKkQFFGZM942rKl8HN/Td/rDSslJZH5wou8\neNAt1N76E4mjb2LK68pDiEh4FCjKCH8zngoyzuMtltEahgxhQ9OT6FT1e67ZPo4/ObzQmVEiIv4o\nUJQRhc14assSPqEHb3EBe6kOH37IiX++y+I9LfKcpw1/RCRcChRlRKAFc/X4nWe5lu/oQFuWciNP\n0jthCZxxRsAZUIXNnBIRyU2BoozIP+OpGnu5q9Io0kjiKl5iLINIIo2JcTdy/0PerOfKlf1fK9Bx\nERF/FCiKKNyaS4HOD/U6ORVbE4wLmc6qKq14OPNOtnU4hdMbLmOIe4xDfYfnqeKakeH/WoGOi4j4\nZWZl/tGpUycrTZMnm8XFmXmTUb1HXJx3PJzzb7ghvOvYokVm3bt7J7ZpY/bxx0Hb6fPlvXb2w+cr\nxocXkXIDSLUQ7rHqURRBoO1CAyWJA53/3HMhXue33+Bf/4JOnWDZMnj6afjuO+jZM2g7w91CtKiV\naUWkfFOgKIKgJbvDOB5oCCjn/D17YORISEqCV16BwYMhLQ2uvx6qFF59JZwNhkLd2EhEKh5tXFQE\nIW0CFML5gfgSjLWPTYMhQ7wLnncejB7tBYwSEu5nEpGyTxsXlaBwh3RCqdGUrWuNRcyv2R0uuQQO\nPhhmz4a33irRIAHh94ZEpOIQTlCjAAAP/0lEQVRQoCiCcPeMzn1+IPX5ldcOupov93am3tYf4dln\nvTxEjx4l8yHyCbROI9BxEak4FCiKKDnZG5LJzPR+FrZdaPb5zuU9XoPdDOUh0kiiz9+Tcbff7uUh\nrr3W74KHkko4h9tLEpGKQ4EiDJG4SR/4hm5cwuusoCUPMZwva54Oy5fDqFFw6KEB37+kEs7h9pJE\npOJQoAhRpG7SKSlwYvVUvuAkXudS/sdhnFX9U7Y8Px2OPjrnvfwFpHCn5YYr3F6SiFQMMTnryTm3\nFtgOZAD7C8vKl8asp4jMCtq4EYYNg4kT2VSpHsMyU/gk4SoeeKhyzk05OyDlDghxcd63+379DuyR\nnZtz3s1dRCQcoc56iuVA0dnMNodyfmkEikqVinGT3r0bHnvMWxOxbx8MGuQFjEMOKXBqsIAEmsIq\nIpGj6bERVqRZQWYwdSq0aAH33OPtLLdihRcw/AQJCD5NVQlnEYmGWA0UBnzsnFvonLs22o2BItyk\n58+Hbt3g8suhdm347DN4802mzGsaNCFeu7b/yyUkKOEsIlESSkGo0n4ADbN+1gOWACf7OedaIBVI\nTUhIiFCJrOAmT/YK6jnn/fRbvG/9erN+/bzqe0ccYfbCC2b79+e8PlgRwMmTzapVK1jEr2rVIIUC\nRUSKiBCLAsZkjiI359wIYIeZPRronNIu4eHXrl3ePtWPPAL793t1mYYN81ZXZyksIR7o+fh42BxS\ntkZEJHRlNkfhnDvIOXdw9r+B04EfSvp9i7xGwgxefdXLQ9x7L/TqBT/+CA8/nCdIQOFlMgI9v3Vr\niG0RESkBMRcogCOAL51zS4D5wHtm9mFJvqG/NRL9+sGNNxbywm+/ha5d4YoroG5dmDsX3ngDjjzS\n7+mFJcRVRkNEYlHMBQozW21m7bIerc2sxOf0+FvIZgbPPBOgZ/HLL9C3L3Tp4o0ZvfQSLFgAJ58c\n9H0KS4hrVpOIxKKYCxTREGjIxyzfquedO2HECGjeHN5803ty1Sr45z+9MatCFDZrSbOaRCQWxXwy\nOxTFTWYH2y/COcjcn+nlIe68EzZsgD59vKR1YmKR31NEJNrKbDI7GlJSClZ1zXbeEd94eYi+faF+\nffjiC3jttZwgoe1DRaS8U6DAG9r5xz/yHmvML0wmmRm/neCNTb388oFFdFm0faiIVAQVMlD46wX8\n9JP3XBw7GcG9rKQ5FzKdxw+928tD9O9fIA9R0tVcRURiQYXLUQSqzrp7VybJTGEkd9GIjbzKZdzF\nSH5xvoBF/4pVKFBEJMqUowjAXy+g3a6v+ZYuTOJKNtCIrnzFFbzKOnwkJATOQxS27kH5CxEpD8pF\nj6J69c62b18qCQleYjrYdNLcvYAE0hnJXVzOVDbQkHurjeTFv5OxrPgZF+eNOE2c6H9/CAi8d0Sw\n5zTdVURiQag9iqgXAIzEAzr5LbLnj89ndhDb7X7utl3UsF3UsBH821o02eG36J/PV7BIH3jHzQIX\nCizsdSIi0UZ5KQoYCuc6m1dI1hNwI5/MTL6+YRJHPj+UBvYrU7iCoTzMlriEgN/0i5qHUP5CRGJd\nhc5R+F1p/eWXcNxxdH3un1RtmsCF9b+mn5tCJV/gIAFFr7+kuk1S3o0ZM4ZWrVrRtm1bevToQXqg\nVatZevfuTZs2bXJ+HzFiBI0aNaJ9+/a0b9+e999/H4D58+fnHGvXrh0zZszIec3VV19NvXr18lyn\npJx55pkcdthhnHPOOX6fv+WWW6hVq5bf54J9hvHjx9OmTRtat27NuHHjco4vXryYLl260L59ezp3\n7sz8+fMBmDlzJm3bts05/uWXX0bwU4YolG5HrD9yDz0VGN5Zs8asTx/viUaNvLGhjIyQu2aF7SER\n6deJlBWffvqp7dy508zMnnrqKevTp0/Ac6dNm2aXX365tW7dOufYvffea6NHjy5w7s6dO23fvn1m\nZrZx40arW7duzu9z5861hQsX5rlOUfhCGAOePXu2vf3223b22WcXeG7BggXWt29fO+igg/y+NtBn\n+P77761169Y5z/fo0cNWrVplZmY9e/a0999/38zM3nvvPevevbuZmW3fvt0yMzPNzGzJkiXWvHnz\ncD9uQIQ49FTuehQ5RfS2b/emOLVoAe+849VoWrnS6zqEUJcpW1HrL6luk8SSe+65h/Hjx+f8Pnz4\ncB5//PFiXfPUU08lLquKZZcuXVi/fr3f83bs2MGYMWO4++67Q7puXFwcVapUAWDPnj24XGUTTj75\nZGrn2wZy//79HHvssXz22WcADB06lOERWMzUo0cPDs63VQBARkYGQ4YMYdSoUWF/hhUrVtClS5ec\n57t3757T23DOsW3bNgD++usvGjZsCECtWrVyXr9z5848f49SE0o0ifVHtWqdDiSTX8kwe/FFs/r1\nva/xffua/fJLhOKvSNm0Zs0a69Chg5mZZWRkWNOmTW3z5s0FzuvWrZu1a9euwGPWrFlBr3/TTTfZ\nAw884Pe5gQMH2vTp023NmjUFehQ+n8+OOeYYu+qqq2zr1q05z33zzTfWqlUrO+igg2z69OkFPkv+\nHsUPP/xgLVq0sI8//tjat29ve/fuDdreUHoUZmZz5swp0KMYN26cjRkzxswsYI8i0GdYvny5JSUl\n2ebNm23nzp3WpUsXu/nmm3Oea9KkiTVu3NgaNmxoa9euzbnW9OnTrXnz5nb44Yfb119/HVLbQ0GI\nPYqo3+Qj8ejUqZP3qefONevY0ftYXbqYffNNJP6WIuXCaaedZosWLbIPPvjALrrooohdd9KkSXb8\n8cfbnj17Cjz33Xff2TnnnGNmBW/wv/32m+3fv98yMjJs2LBhdtVVVxV4/fLly+3YY4+13bt35xzz\nFyjMzFJSUqx69eq2aNEiv+188MEHcwJf1apVc/594403Bvxs+QPFhg0b7MQTT8wZVgoWKAJ9hgkT\nJliHDh3spJNOsuuuu84GDhxoZma33HKLvfnmm2Zm9tprr1mPHj0KXGvu3Ll+jxdVxQoUbdqYXXyx\n93GaNDH773/Nssb0RMQzdepUu/XWW61Pnz723nvv+T0n3B7FrFmzrEWLFvb777/7ff6pp56yBg0a\nmM/ns0aNGlnVqlVzxt5zC3TzNzM75ZRTbMGCBYWee9lll9kRRxxhH3zwgd/r5FbUHsW7775rRxxx\nhPl8PvP5fOacs6OOOqrQ6+T/DNmGDh1qTz75pJmZHXLIITm5iMzMTDv44IP9XisxMdE2bdoUUvsL\nU7EChXNepvj++8127gy4tkGkItu7d681a9bMjjzySNu/f3+xr7do0SJr2rRpTjK2MPlv8Bs3bsz5\n95gxY+zSSy81M7PVq1fnfGNfu3atNWjQIM+N0V+gmDZtmvXs2dNWrlxpSUlJ9ueffwZtS3GGnnIL\n1KMI9hmyg2p6ero1b948Z8itRYsWNmfOHDPzEukdO3Y0M7O0tLScALJw4UJr2LBhzu/FVbECRe3a\nZuvXm1ng2UY33KDgIXLdddfZnXfeGZFr9ejRw+rVq5fT6zj33HNznmvXrl2B8/Pf4Pv27Wtt2rSx\nY445xs4999ycwPHKK69Yq1atrF27dtahQwebMWNGzmsuu+wyq1+/vlWpUsUaNWpkEyZMsE2bNllS\nUpKtW7fOzMzGjx9vV155ZdC2hxIounXrZnXq1LEaNWpYo0aN7MMPPyxwTu5AMXPmTLvnnnsK/Qzd\nunWzli1bWtu2bW327Nk5x7/44gvr2LGjtW3b1o477jhLTU01M7ORI0fmXKtLly72xRdfFNr2UIUa\nKMrFgrvcRQEDbULkXN4FcCqnIRVNZmYmHTt25I033iApKSnazZEYUGEX3AXb1jQ3lQOXimT58uUc\nffTR9OjRQ0FCwlYl2g2ItISEwNua5hcoqIiUN61atWL16tXRboaUUeWuR5GS4g0r5RZofYrKaYiI\nFK7cBQp/K6Kvv75g8MhZwS0iIkGVu0ABXrBYu9ar0rp2LTz1lMppiIgUVbnLUQSSnKzAICJSFOWy\nRyEiIpGjQCEiIkEpUIiISFAKFCIiEpQChYiIBFUuaj055zYBIa7HLjF1gM1RbkOs0N/iAP0tDtDf\n4oBY+Vv4zKxuYSeVi0ARC5xzqaEU16oI9Lc4QH+LA/S3OKCs/S009CQiIkEpUIiISFAKFJHzXLQb\nEEP0tzhAf4sD9Lc4oEz9LZSjEBGRoNSjEBGRoBQoSoBz7nbnnDnn6kS7LdHinBvtnPvRObfUOTfD\nOXdYtNtU2pxzZzrnVjrnfnLO3RXt9kSLc66Jc26Oc26Fc26Zc25AtNsUbc65ys6575xz70a7LaFQ\noIgw51wToCdQ0ffPmwW0MbO2wCpgaJTbU6qcc5WBJ4GzgFbA5c65VtFtVdTsB24zs5ZAF+CmCvy3\nyDYAWBHtRoRKgSLyxgJ3ABU6+WNmH5vZ/qxfvwEaR7M9UXAc8JOZrTazv4GpwHlRblNUmNmvZrYo\n69/b8W6QjaLbquhxzjUGzgYmRLstoVKgiCDnXG9gg5ktiXZbYszVwAfRbkQpawT8kuv39VTgm2M2\n51wi0AH4NrotiapxeF8mM6PdkFBVmI2LIsU5Nxuo7+ep4cAw4PTSbVH0BPtbmNnMrHOG4w09TCnN\ntsUAfzu1V+hepnOuFjANGGhm26Ldnmhwzp0D/GFmC51zp0S7PaFSoAiTmZ3m77hz7hjgSGCJcw68\noZZFzrnjzOy3UmxiqQn0t8jmnOsPnAP0sIo3D3s90CTX742BjVFqS9Q556riBYkpZjY92u2JohOB\n3s65XkAN4BDn3GQz6xvldgWldRQlxDm3FuhsZrFQ+KvUOefOBMYA3c1sU7TbU9qcc1Xwkvg9gA3A\nAuAKM1sW1YZFgfO+OU0EtprZwGi3J1Zk9ShuN7Nzot2WwihHISXlCeBgYJZzbrFz7ploN6g0ZSXy\nbwY+wkvevl4Rg0SWE4F+wD+y/i8szvpGLWWEehQiIhKUehQiIhKUAoWIiASlQCEiIkEpUIiISFAK\nFCIiEpQChYiIBKVAISIiQSlQiIhIUAoUIiISlAKFSAQ556o4575yzu1wzrXI99y1WTsf3h+t9okU\nhUp4iESYc84HLAbSgS5mtsc51xqvMOBC4BQzy4hmG0XCoR6FSISZWTpwDdAOeNQ5VxNvh7s9QLKC\nhJQ16lGIlBDn3FPADcDXQFfgogq+F4OUUQoUIiXEOVcD+AE4CnjezK6NcpNEikRDTyIlpy2QkPXv\nNlmbGYmUOQoUIiXAOXcIXl5iM95+6icA90W1USJFpG84IiXjWcAH9DSzT51z7YG7nHOzzWxOlNsm\nEhblKEQizDl3DTABeMjMhmcdOwxvymxVoK2ZbYliE0XCokAhEkFZi+wW4gWF7ll7Z2c/dwLwOfCB\nmfWOUhNFwqZAISIiQSmZLSIiQSlQiIhIUAoUIiISlAKFiIgEpUAhIiJBKVCIiEhQChQiIhKUAoWI\niASlQCEiIkEpUIiISFD/D7VBLa+6XfjJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x8b5cd70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 请不要修改下面的代码\n",
    "x1,x2 = -5,5\n",
    "y1,y2 = x1*m+b, x2*m+b\n",
    "\n",
    "plt.xlim((-5,5))\n",
    "plt.xlabel('x',fontsize=18)\n",
    "plt.ylabel('y',fontsize=18)\n",
    "plt.scatter(X,Y,c='b')\n",
    "plt.plot((x1,x2),(y1,y2),'r')\n",
    "plt.text(1,2,'y = {m}x + {b}'.format(m=m,b=b))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你求得的回归结果对当前数据集的MSE是多少？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.890985275344\n"
     ]
    }
   ],
   "source": [
    "mse = calculateMSE(X,Y,m,b)\n",
    "print(mse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
